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CONTACT  MANAGEMENT  MODEL  ASSESSMENT: 
AN  APPROACH  AND  SYSTEM  DESCRIPTION 


1.  INTRODUCTION 


Tracking  acoustic  sources  in  the  ocean  environment  remains  a  difficult  problem  and 
continues  to  be  an  area  of  active  research. In  the  contact  management  tracking  problem,  noise 
corrupted  sonar  measurements  are  used  to  obtain  estimates  of  the  contact's  state  parameters 
consisting  of  range,  bearing,  depth,  course,  and  speed.  The  fundamental  physical  quantities 
measured  by  a  sonar  system  are  time  delay  and  frequency.  The  time  delay  from  spatially  diverse 
sensor  elements  is  often  converted  to  angle  of  arrival  information,  examples  of  which  are 
azimuthal  bearing  or  conical  angle  depending  on  the  sensor  employed.  State  estimation  in 
nonlinear  dynamic  systems  of  this  type  requires  the  data  to  be  processed  under  various  kine-matic, 
environmental  and  sensor  modeling  assumptions.^  ’ 

A  typical  state  estimation 
process  is  depicted  schematically 
in  figure  1 .  As  shown,  the 
residual  error  in  the  fit  to  the 
measurements,  or  simply  the 
residuals,  is  the  difference 
between  the  actual  measurements 
and  an  estimate  of  the  measure¬ 
ments  obtained  from  the  state 
estimate  and  the  system  model. 
Using  the  residuals,  the  gradient 
relation  maps  the  error  in  the  fit 
to  the  data  into  a  correction  term 
Figure  1.  State  Estimation  Process  for  the  state  estimate.  The 

gradient  relation  is  based  on  the 
mathematical  system  model  that 

relates  the  state  to  the  measurements.  The  state  estimate  is  updated  with  the  correction  term,  new 
residuals  are  computed  and  the  process  is  iterated  until  the  estimation  algorithm  converges  to  a 
state  estimate  that  provides  a  "best-fit,"  by  some  measure,  to  the  observed  measurements.  Thus, 
because  the  estimated  measurements  and  gradient  relations  are  dependent  on  the  model  of  the 
system,  the  judicious  selection  of  appropriate  models  is  critical  if  accurate  state  estimates  are  to  be 
obtained.  When  the  system  model  sufficiently  reflects  the  actual  process  by  which  the 
measurements  were  produced,  the  residuals  are  noise-like  in  character.  That  is,  they  are  devoid  of 
any  deterministic  features. 

Often  the  correct  models  are  not  known  a  priori  and  must  be  determined  along  with  the 
contact  state.  Additionally,  the  system  must  also  adapt  to  changing  conditions  of  the  scenario, 
that  is,  changes  in  the  system  model.  Traditionally,  these  conditions  are  accommodated  using  a 
combination  of  batch  or  sequential  state  estimation  techniques  with  multiple  hypothesis  test 
procedures.*  In  these  approaches,  all  possible  models  are  used  to  obtain  state  estimates  and  the 
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solution(s)  with  the  best  residuals  or  largest  likelihood  are  selected  as  the  correct  model(s).  As 
these  model  assessment  methods  apply  all  to  models  simultaneously,  large  amounts  of  compu¬ 
tational  resources  may  be  required  to  achieve  real  time  performance.  Further,  because  some  of 
the  possible  models  are  inconsistent  with  the  observed  data,  the  estimation  algorithms  may  fail  to 
converge.  As  a  consequence,  nontraditional  methods  for  state  estimation  in  the  presence  of 
model  ambiguity  and  uncertainty  are  being  explored  for  application  to  the  contact  management 
model  assessment  (CMMA)  problem. 

It  is  well  known  that  changes 
in  the  underlying  process  can 
induce  identifiable  features  into 
predicted  residual  sequences.  A 
nontraditional  processor  for  model 
assessment,  which  took  advantage 
of  this  property,  was  originally 
proposed  in  Baylog,  et  al.^  and 
extended  in  Ferkinhoff,  et  al.^’"* 

The  system,  which  continues  to  be 
developed,  is  illustrated  in  its 
currently  implemented  form  in 
figure  2.  The  approach  uses  state 
estimation  and  hypothesis  testing, 
coupled  with  Dempster-Shafer 
(DS)  evidential  reasoning,"  '^  to 
resolve  modeling  ambiguity.  As  different  classes  of  modeling  errors  can  induce  unique  features  in 
residual  sequences  of  the  various  measurement  types,  it  is  often  possible  in  theory  to  deduce  the 
type  of  modeling  error,  or  at  least  identify  the  class  to  which  it  belongs.  Previous  reports  have 
discussed  the  extraction  and  representation  of  evidence  as  well  as  the  overall  performance  of  the 
DS  theory  approach  to  the  CMMA  problem.^’*”  In  those  studies,  it  was  assumed  that  modeling 
consistency  had  been  achieved  and  that  a  change  of  the  model  subsequently  occurred.  However, 
the  system  is  not  limited  to  that  application. 

This  report  discusses  several  advances  in  the  application  of  the  DS  theory  of  evidential 
reasoning  to  the  CMMA  problem.  A  hypothesis  selection  criterion  for  the  DS  frame  of  reasoning 
is  developed.  The  approach  uses  a  geometric  interpretation  of  the  DS  frame ,  which  in  itself  can 
be  a  useful  tool.  With  the  development  of  the  hypothesis  selection  criterion,  the  previous  results 
are  extended  to  include  the  efficient  selection  of  models  for  further  processing.  Novel  evidence 
representation  functions  that  map  evidence  produced  by  probabilistic  multiple  hypothesis  tests 
into  belief  in  the  deterministic  features  are  developed.  A  two-stage  approach  is  employed  that 
first  establishes  the  belief  in  the  existence  of  a  feature  and  then  refines  that  belief  to  one  of  feature 
amplitude.  This  extension  of  the  evidence  representation  allows  for  a  greater  degree  of 
discernment  which  leads  to  higher  resolution  of  the  possible  modeling  hypotheses.  These 
advances  have  been  reported  separately  elsewhere®’*®’'^’*'*  but  are  collectively  presented  here  with 
the  CMMA  application  for  the  first  time. 


Figure  2.  Block  Diagram  of  CMMA  System 
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The  remainder  of  the  report  is  organized  as  follows:  section  2  reviews  the  DS  theory  of 
evidential  reasoning  so  that  the  application  to  CMMA  can  be  presented  in  the  proper  context. 
Included  in  this  section  are  the  geometric  representation  of  the  DS  frame  and  the  hypothesis 
selection  criterion;  section  3  details  the  application  of  the  DS  theory  to  the  CMMA  problem. 

Also  included  in  this  section  are  the  general  overview  of  the  structure  of  the  evidential  reasoning 
process  and  the  development  of  the  evidence  representations.  Experimental  results,  obtained  with 
simulated  data,  are  presented  and  discussed  for  the  CMMA  problem  in  section  4  and  section  5 
contains  conclusions  with  suggestions  for  future  work.  Several  appendices  are  included  that 
provide  details  of  specific  developments. 


2.  DS  EVIDENTIAL  REASONING 


The  DS  theory  of  evidential  reasoning  is  one  of  several  approaches  for  producing  inferences 
from  uncertain  information.  Its  appeal  for  application  to  model  assessment  is  that  it  intrinsically 
accommodates  the  expression  of  ignorance  and  naturally  provides  a  convenient  framework  on 
which  the  CMMA  problem  can  be  structured.  Section  2. 1  reviews  the  basic  mathematical  theory 
of  DS  evidential  reasoning  (DS-ER).  A  geometric  representation,  which  is  not  a  standard  part  of 
DS-ER,  is  presented  in  section  2.2.  This  representation  is  used  in  section  2.3  in  the  development 
of  a  hypothesis  selection  criterion. 


2.1  DS  THEORY 

The  basic  structure  for  DS-ER  is  the  frame  of  discernment.  Denoted  by  0,  the  frame  is  a  set 
of  mutually  exclusive  and  exhaustive  hypotheses: 

0^  ={a,...a^}.  (1) 

The  power  set ,  denoted  by  2®)  is  the  set  of  all  subsets  of  the  frame  of  discernment  and  has 

K  =  2'®'  - 1  (2) 

elements,  where  |01  is  the  cardinality  of  the  set  0.  The  minus  one  accounts  for  the  null  set  0 
which  is  not  considered  a  member  of  the  power  set  in  DS  theory.  Note  that  the  cardinality  of  the 
power  set  grows  exponentially  with  the  number  of  hypotheses.  Figure  3  illustrates  a  generic 
power  set. 
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{a0^a1|[2a3} 


a0,a1,a2}  {a0,a1,a3}  {a0,a2,a3}  {a1,a2,a3 


{aO,a1}  {a0,a2}  {aO,a3}  {a1,a2}  {a1,a3}  {a2,a3) 


{aO>  {a1>  {a2}  {a3} 


Figure  3.  Power  Set 

A  basic  probability  assignment  (BP  A),  denoted  m{Aj),  is  a  function  that  assigns  a  probability  to 
each  element  of  the  power  set,  Aj,  representing  the  belief  that  directly  supports  the  truth  of  the 
hypotheses  contained  in  the  set  Aj .  The  individual  BP  As  are  bounded  between  zero  and  one  and 
sum  to  unity.  It  is  noted  here  that  BPA  has  a  special  connotation  in  that  it  is  defined  to  be  beliefs 
assigned  to  the  initial  power  set.  While  belief,  which  is  generated  in  later  stages  of  the  system,  is 
produced  by  operating  on  the  original  BPA,  and  is  also  referred  to  as  m{Ai),  this  belief  is  not 
generally  referred  to  as  BPA,  only  as  belief  However,  the  belief  distributed  among  the  power 
sets  in  the  latter  sages  of  the  system  must  still  conform  to  the  constraints  imposed  on  the  original 
BPA,  i.e.,  they  are  bounded  between  zero  and  one  and  sum  to  unity  over  any  power  set. 
Additional  measures  of  plausibility  and  support  for  each  element  can  also  be  computed.  The 
support  for  Aj,  denoted  by  s{Aj),  is  given  by  the  sum  of  the  belief  over  all  the  subsets  of  Aj  as 

^m{a),  (3) 

OQ  Ai 

and  represents  the  belief  that  directly  supports  the  hypotheses  contained  in  Aj.  The  plausibility  of 
Aj,p{Aj),  is  a  measure  of  the  lack  of  support  in  the  complement  ofAj  and  is  given  by  one  minus 
the  support  in  the  complement  of  Aj  as 

P{A,)=  1.0-5(4)=  (4) 

ar\Ai^  0 

Note  that  in  general,  support  is  less  than  or  equal  to  plausibility  and  both  are  bounded  by  zero  and 
one.  The  uncertainty  in  assigning  belief  to  the  hypotheses  contained  in  element  Aj  is  the 
difference  between  the  plausibility  and  support.  Note  that  DS  theory  includes  Bayesian 
probability  as  a  special  case  when  all  belief  is  distributed  among  the  singleton  sets  {aj},{a2},.... 
Examples  of  computing  plausibility  and  support  are  given  at  the  end  of  this  section  and 
section  2.3. 
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If  several  bodies  of  evidence  exist  for  a  common  frame  of  discernment,  the  resulting  beliefs 
can  be  combined  using  Dempster’s  combination  rule, 

w,(4)=  —  Z  (5) 

^  Ajr\Ai[~Ai 

where  m\{Aj)  and  /W2(^*)  are  the  belief  in  Aj  and  Ak  generated  from  two  bodies  of  evidence,  Ei 
and  E2,  represented  in  two  common  frames  of  discernment  0i  and  02,  respectively.  The  term  a  is 
the  renormalization  constant  necessary  to  account  for  belief  being  placed  into  the  null  intersection 
0  and  is 

a  =  \-  Yu  WiKH(A)-  (6) 


In  both  expressions,  the  indices  j  and  k  range  over  all  the  elements  of  the  power  set.  Dempster's 
combination  rule  is  both  commutative  and  associative.  An  example  using  Dempster's  combination 
rule  is  given  at  the  end  of  this  section. 

Dempster's  combination  rule  is  valid  only  for  identical  frames  of  discernment.  However,  it  is 
often  the  case  that  multiple  bodies  of  evidence  exist  for  dissimilar  but  related  frames.  To  combine 
evidence  from  different  frames  of  discernment  it  is  necessary  to  map  them  to  a  common  frame. 
This  mapping  is  called  a  compatibility  map  and  is  a  logical  relationship  (i.e.,  one  or  zero)  between 
the  hypotheses  of  one  frame  (A)  and  the  hypotheses  of  another  frame  (B).  Thus,  compatibility 
maps,  which  are  used  to  map  belief  in  one  frame  of  discernment  to  a  different  frame,  are  useful  for 
combining  belief  that  originates  in  dissimilar  frames  into  belief  in  a  common  frame.  The 
compatibility  map  defines  the  relationship  between  the  elements  of  two  frames  0a.b  where 

&A.B  ^0aX0b.  (7) 

and  the  compatibility  mapping  is 

=  (8) 

where  at  least  one  pair  (aj.bj)  is  specified  for  each  of  the  Ak  in  0^.  The  BPA  for  an  element  of  the 
power  set  of  the  new  frame,  B/  is  obtained  by  assigning  it  the  sum  of  BPA  over  all  the  compatible 
Ak. 


Dempster's  combination  rule  and  the  compatibility  relationship  are  perhaps  best  explained  by 
way  of  an  example.  Consider  two  dissimilar,  but  compatible,  frames  of  discernment 
Qa  =  ^^^sic  probability  assignments: 
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/w,(4)  =  /w,({ao,a,})  =  0.1 

/w,(4)  =  /w,({ao,«2})  =  0.1 
/M,(.4J  =  »i,({a,,a2})  =  0,0 
/w,(4)  =  /w,({ao})  =  0.3 
m^{A^)  =  m,{{a,))  =  05 
/M,(^7)  =  /w,({a2})  =  0.0 


/n2(5,)  =  W2({Z)o,^,*2})  =  0.2 
W2(57)  =  /W7({^>o,Z>,))  =  0.4 
/W2(53)  =  W2({6o,M)  =  0.3 

'«2(^6)  =  "*2({^})  =  0.0 
m7(57)  =  /n7({Z>7})  =  0.0 


Further,  suppose  that  the  compatibility  relation  of  A^B  is  given  by: 


bo 

bl 

hi 


That  is,  the  compatibility  relation  for  the  singletons  is  given  by: 

C,4/I,  =  {a,})  =  {i,) 

=  {«,))  =  {*.) 

^A-*b{As  =  {^o})  “ 


Because  the  compatibility  relation  of  a  union  of  sets  equals  the  union  of  the  compatible  subsets, 
Ca~.b{A  ^  =  Ca^b(A)^Ca-.b(Aj) 

it  follows  that 

~  {^1  ’^2})  ~  -^e)  ”  (A)>^i  }  ’ 

~  {^0>^2  })  “  (^0>  A  }  J 
C^->b(^2  =  K.«i})  =  ,  and 
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The  compatibility  relationship  is  illustrated  in  table  1  below. 


Table  1.  Compatibility  Relationship 


From  the  table,  it  is  seen  that  the  basic  probability  assignments  in  the  frame  0b  from  the  first  body 
of  evidence  are: 

m^[{b^,b^,b^'^  =0.0, 

^■({^0.^2})  =0.0, 

"».({6i.*2})  =0.0, 

w,({*o})  =  "»i({«i})  =0.5, 

^i({^})  =  '”i(k})  =0.0, 

"»i({*2})  =  0.0, 

where  the  subscript  mj  has  been  retained  to  differentiate  the  first  body  of  evidence  from  the 
second.  Now  that  the  two  bodies  of  evidence  are  represented  in  the  same  frame,  Dempster's 
combination  rule  can  be  used  to  combine  the  two  independent  frames  to  a  single  solution  frame. 
Dempster's  combination  rule  is  illustrated  in  table  2  below  where  the  elements  with  zero  belief  are 
not  shown. 


Table  2.  Dempster's  Combination  Rule 


mi\m2 

m2(QB)  =  0.2 

/W2({Mi})  =  0.4 

m2({bob2})  =  0.3 

W2({*o})  =  0.1 

mi({bo,bi})  =  0.5 

Mi\o.i 

/^obi\0.2 

^>o\0.15 

^»o\0.05 

m^{{bo})  =  0.5 

^>o\0.1 

6o\0.2 

Ao\0.15 

6o\0.05 

Here,  the  rows  correspond  to  the  elements  of  the  power  set  of  the  first  frame  while  the  columns 
correspond  to  the  second  frame.  Each  cell  in  the  table  is  associated  with  a  member  of  the  power 
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set  in  the  combined  frame  given  by  the  intersection  of  the  sets  associated  with  the  corresponding 
row  and  column,  and  the  belief  is  the  product  of  the  BP  A  of  the  individual  sets.  Only  the  non¬ 
zero  entries  are  shown.  The  entries  in  each  cell  are  the  elements  of  the  power  set  and  the 
resulting  portion  of  BP  A  to  be  assigned  to  that  element.  As  there  are  no  null  intersections, 
a  =  1  by  equation  (6).  Using  equation  (5),  the  belief  values  in  table  2  are  combined  resulting  in; 


/m^({Z»o,^i})  =  0.3, 

mc{,{bo})  =  0.7. 


Using  equations  (3)  and  (4),  the  support  for  bo  is  0.7,  the  plausibility  is  1.0,  and  the  uncertainty  is 
0.3,  while  the  support  for  is  equal  to  the  plausibility  for  which  is  1.0. 

With  a  network  of  compatibility  relations,  different  frames  of  reasoning  can  be  linked 
together  and  combined.  The  collection  of  frames  and  compatibility  relations  and  the  place  where 
they  are  combined  is  called  a  gallery.  It  is  noted  that  differing  galleries  can  be  used  for  the  same 
problem,  and  that  in  general  each  gallery  can  behave  differently  than  another. 


2.2  GEOMETRIC  REPRESENTATION  OF  DS  THEORY 

Consider  the  generic  frame  of  discernment  comprising  the  set  of  N  mutually  exclusive  and 
exhaustive  hypothesis,  0^,  as  given  in  equation  (1)  and  repeated  here  for  convenience  as  equation 

(9), 


0^  =  (9) 

As  previously  noted,  there  are  three  useful  measures  of  the  elements  of  the  power  set  Af  the 
BP  A,  support,  and  plausibility.  Although  the  interrelation  of  these  three  quantities  is  often 
complex  for  large  frames,  some  insight  may  be  obtained  with  a  geometric  representation  of  these 
measures. 

Let  each  element  of  the  power  set  be  represented  by  a  point  in  the  three  dimensional  (space 
of  real  numbers  with  coordinates  of  plausibility,  support,  and  belief,  denoted  by 


AA;)  =  \p{A),siA;),m{A;)l  (10) 

Because  it  is  necessarily  true  that 

piAi)>s{A^>m{A;),  (11) 


the  space  of  allowable  points  that  represents  the  elements  of  the  power  set  is  restricted  to  the 
interior  and  surface  of  the  solid  (right)  triangle  or  wedge  of  unit  height  shown  in  figure  4. 


Referring  to  figure  4,  the  following  discussion  illustrates  the  relationship  between  the 
distribution  of  BP  A  in  the  frame  and  the  measures  of  support  and  plausibility.  The  terminology 
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introduced  has  been  adopted  to  facilitate  discussion  and  should  not  be  considered  as  a  standard. 
The  point  (1,1,1)  is  the  certainty  point.  That  is,  any  element  ^4;  that  is  represented  by  this  point 
has  all  the  belief  assigned  directly  to  it,  and  therefore,  also  has  a  support  and  plausibility  of  one. 
If  BPA  is  redistributed  from  the  element  itself  to  its  proper  subsets,  then  its  BPA  is  reduced 
while  both  plausibility  and  support  remain  at  a  value  of  one.  Its  representational  point  moves 
down  the  belief  line  ((l,l,0)-(  1,1,1))  until  all  the  belief  has  been  redistributed  exclusively  to  its 
proper  subsets.  In  that  case 


CERTAINTY  POINT 


Figure  4.  Geometric  View  of  Dempster-Shafer  Frame 


there  is  no  belief  in  the  element  itself  and  the  coordinate  location  is  the  point  (1,1,0).  This  point 
has  been  labeled  the  inclusion  point  because  although  it  has  zero  BPA,  its  subsets  include  all  the 
available  support  and,  therefore,  must  in  theory  include  the  correct  hypothesis.  If  BPA  is  now 
redistributed  from  the  proper  subsets  into  sets  that  are  not  subsets  but  have  non-empty 
intersections  with  Aj,  then  support  for  Aj  is  reduced  while  its  plausibility  remains  one.  The  point 
moves  along  the  support  line  ((l,0,0)-(  1,1,0))  until  the  point  (1,0,0)  is  reached.  This  point  has 
been  labeled  the  indismissible  point.  Although  no  evidence  supports  the  hypotheses  of  this 
element,  there  is  no  evidence  to  the  contrary.  Thus,  the  plausibility  is  still  high  and  there  is  no 
reason  to  dismiss  these  hypotheses.  Further  redistribution  of  BPA  into  the  compliment  of  and 
its  subsets  reduces  plausibility.  The  resulting  movement  is  along  the  plausibility  line  ((0,0,0)- 
(1,0,0))  until  the  point  (0,0,0)  is  reached.  At  this  point,  all  the  belief  is  distributed  among  the 
subsets  in  the  complement  ofAf.  This  point  has  been  labeled  as  the  exclusion  point  because  all 
available  BPA  is  in  the  compliment,  thus  these  hypotheses  may  be  excluded.  Other 
redistributions  of  belief  to  the  subsets,  supersets,  intersecting  sets,  and  compliment  sets  result  in 
points  located  within  or  on  the  surface  of  the  wedge.  Note  that  more  than  one  subset  of  the  frame 
may  be  located  at  a  given  point  in  the  solid  triangle.  However,  there  are  some  restrictions  such 
as  only  («-l)  elements  of  the  power  set  may  be  located  above  the  level  of  belief  with  value  (1/n). 
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2.3  HYPOTHESIS  SELECTION  CRITERION  FOR  DS  THEORY 


This  section  provides  the  details  of  the  proposed  hypothesis  selection  process  for  the  DS 
frame  of  ER.  Referring  to  figure  4,  the  proposed  method  projects  all  the  points  into  the  plane  of 
maximum  plausibility,  or  the  support-belief  plane.  A  belief  cut  is  defined  as  a  plane  of  constant 
belief,  which  becomes  a  line  of  constant  belief  when  projected  into  the  support-belief  plane. 
Starting  with  the  set(s)  with  the  highest  value  of  BP  A,  the  union  of  all  subsets  represented  by 
points  on  the  belief  cut  is  formed  and  the  support  of  the  resulting  union  computed.  The  process  is 
repeated  at  the  next  highest  level  of  belief,  again  taking  the  union  of  all  the  sets  with  BPA  equal  to 
or  greater  than  the  belief  cut  value  and  computing  its  support.  The  process  is  continued  until 
some  minimum  threshold  in  support  is  met  or  exceeded.  The  final  union  contains  all  the 
hypotheses  for  fiirther  consideration.  Although  this  approach  may  appear  to  be  involved,  this  is 
not  necessarily  the  case,  since  only  those  elements  of  the  power  set  with  non-zero  BPA  and  their 
unions  need  to  be  enumerated.  This  can  often  yield  a  significant  computational  savings.  Note 
that  although  the  method  results  in  an  unambiguous  selection,  it  is  possible  that  other  elements  of 
the  power  set  have  the  same  support,  however,  the  selected  set  will  be  a  proper  subset  of  these 
other  elements. 

The  hypothesis  selection  method  is  equivalent  to  ranking  the  elements  of  the  power  set  by 
BPA,  then  forming  the  union  of  subsets  in  order  of  decreasing  BPA  until  a  threshold  on  support 
of  the  union  is  met  or  exceeded.  Specifically,  let  B  be  the  ordered  set 

(12) 

such  that 

miB-^tm{Bj)  ,  j>i;  (13) 

and  where  K  is  the  number  of  elements  in  the  power  set  as  previously  defined.  Let  nid,  d  =  1,2,. . . 
be  the  distinct  values  of  belief  where  ntj  >  ntd-u  and  nj  the  number  of  sets  with  belief  ntd- 

Let  the  number  of  sets  from  each  of  the  q  belief  cuts  be  Lg,  where 


^=1,2,... 


(14) 


Define  the  union  of  sets  of  B ’s  as  Cj 

C/  =  U^«.  (15) 

i-i 

The  selection  criterion  is  then  simply  the  set  Cz,^  with  the  smallest  value  of  q,  such  that  its  support 
meets  or  exceeds  the  threshold  X, 

minsiCr)  ^  (16) 

A 
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Since  DS  representation  in  computer  software  is  most  efficiently  implemented  by  tracking 
only  those  subsets  of  the  frame  with  non-zero  belief,  no  additional  elements  need  to  be 
represented  beyond  those  resulting  strictly  from  application  of  the  gallery,  other  than  possibly  a 
few  that  result  from  the  union  operation  given  in  equation  15.  Further,  since  the  sets  are 
combined  in  order  of  decreasing  belief,  support  for  the  union  of  the  sets  grows  in  the  most 
efficient  manner. 

To  evaluate  the  selection  criterion,  a  limited  set  of  experimental  results  were  obtained  for  a 
generic  frame  with  four  elements.  The  specific  elements  with  non-zero  BP  A,  as  well  as  the 
number  of  elements  with  non-zero  BP  A,  and  their  values,  were  selected  from  a  uniform 
distribution.  The  power  set  is  enumerated  in  table  3  with  a  selected  set  of  BP  A  and  the  resulting 
measures  of  support  and  plausibility.  The  entries  have  been  rounded  for  simplicity. 

Table  3.  Frame  of  Discernment 


j _ 

0.80 

0.40 

0.88 

0.35 

0,40 

0.68 

0.15 

0.28 

0.12 

0.23 

0.28 

012 

003 

0 


Arranging  the  subsets  of  the  frame  in  order  of  decreasing  (and  non-zero)  BPA  produces  the 
set  ranking,  {a\,a'i),  {a3},  {a0,al),  {a2},  and  {al}  with  BPA  of  0.37,  0.28,  0.20,  0.12,  and 
0.03,  respectively.  Applying  the  selection  algorithm,  the  union  formed  at  the  successive  level  cuts 
are  {al,a3}  with  support  0.68,  {a0,al,a3}  with  support  0.88,  and  {aQ,a\,a2,a3}  with  support 
1 .0.  If  the  support  threshold  is  set  above  the  value  0.68  but  below  0.88,  then  the  selected  subset 
of  hypotheses  would  be  {a0,al,a3 }.  For  the  example  here  only  five  sets  have  non-zero  BPA  and, 
in  combination  with  their  unions,  require  that  only  two  out  of  a  total  of  1 5  support  values  must  be 
computed  for  a  the  given  support  threshold. 

Several  other  selection  criteria  for  the  subsets  of  the  DS  frame  that  involved  distance 
measures  were  examined.  One  such  measure  is  the  distance  of  a  subset  from  the  certainty  point 
(1,1,1).  In  this  method  subsets  are  collected  (unioned)  until  the  accumulated  belief  exceeds  some 


\ _ 

\ _ 

0.97 

0.88 

0.72 

0.77 

088 

0.72 

0.85 

0.32 

0.60 

0.65 

0.12 

0.60 

0.20 
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threshold.  Because  the  frame  0^  is  located  on  the  belief  line  ((l,l,0)-(  1,1,1)),  it  is  always 
included  once  the  radius  has  reached  a  value  of  one.  Consequently,  if  the  belief  threshold  has  not 
been  reached  before  the  frame  is  encountered,  a  complex  set  of  logic  must  be  invoked.  In 
addition  to  the  logic  problem,  all  the  elements  in  the  power  set  must  be  represented,  which  as 
noted  previously,  can  be  a  large  number  for  frames  of  modest  size.  Another  method  examined  the 
distribution  of  points  when  projected  down  into  the  plane  of  no  belief  or  the  plausibility-support 
plane.  Unfortunately,  this  technique  is  subject  to  the  same  limitations  just  mentioned. 
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3.  CONTACT  MANAGEMENT  MODEL  ASSESSMENT 


As  stated  previously,  the  basic  problem  in  contact  management  model  assessment  is  to 
determine  the  set  of  models  that  are  consistent  with  the  observed  data  and  to  obtain  the 
corresponding  contact  state  estimates  along  with  regions  of  uncertainty.  The  specific  problem 
being  considered  here  is  the  case  of  a  change  in  the  dynamic  process  from  a  known  process.  In 
this  class  of  problems,  a  single  correct  model  has  been  identified  and  used  to  obtain  a  good 
contact  state  estimate.  At  a  subsequent  point  in  time,  the  dynamic  process  or  environment 
changes.  The  task  is  to  determine  which  subset  of  models  may  be  consistent  with  the  new  data 
without  actually  processing  the  data  to  obtain  a  state  estimate.  Primarily,  this  is  accomplished  by 
examining  the  predicted  measurement  residual  sequences  for  deterministic  features  that  are 
evidence  of  modeling  errors.  DS  theory  of  evidential  reasoning  is  used  to  determine  which  of  the 
model  hypotheses  could  possibly  have  produced  the  observed  features.  Section  3.1  describes  the 
particular  CMMA  scenario  under  consideration.  The  application  of  DS  evidential  reasoning 
process  to  CMMA  is  given  in  section  3.2.  The  details  of  the  functions  used  to  generate  and  map 
evidence  from  the  predicted  residuals  into  belief  in  the  frame  of  discernment  are  discussed  in 
greater  detail  in  sections  3.3  and  3.4. 


3.1  CMMA  APPLICATION 

The  CMMA  application  considered  here  is  the  tracking  of  a  single  dynamic  contact  by  an 
observer  with  data  from  two  spatially  diverse  acoustic  sensors  in  an  ocean  environment.  One 
sensor  provides  measurements  of  azimuthal  bearing  {^a),  depression  angle  (DE),  and  frequency. 
The  azimuthal  bearing  is  the  angle  of  signal  arrival  in  the  horizontal  plane  and  the  DE  is  the  angle 
of  signal  arrival  in  the  vertical  plane.  The  other  sensor  is  a  line  array  that  provides  measurements 
of  conical  angle  and  frequency.  The  conical  angle  is  the  angle  of  signal  arrival  relative  to  the 
array  axis  and  is  a  function  of  both  the  azimuthal  bearing  and  the  depression  angle. 

In  one  approach  to  contact  management  tracking,  the  data  are  segmented  by  a  statistical 
hypothesis  test  and  compressed  to  an  approximate  sufficient  statistic.  The  compressed  parameters 
are  then  subsequently  processed  to  obtain  a  contact-state  estimate.’^**’  Usually  the  data  are 
segmented  according  to  own  ship  legs.  However,  sometimes  the  cause  for  a  segmentation  in  the 
data  is  a  change  in  the  kinematic,  sensor,  or  environmental  structure  of  the  process.  Thus,  there 
may  be  a  need  to  test  the  data  for  consistency  with  the  model,  prior  to  processing.  For  the  class 
of  problems  considered  here,  the  data  prior  to  the  time  of  a  change  in  the  structure  of  the  problem 
are  processed  using  the  correct  models  to  obtain  a  maximum  likelihood  estimate  of  the  contact's 
state.  A  subsequent  segment  of  data,  possibly  produced  by  a  different  process  and,  thus 
constituting  a  change  in  the  model,  is  assessed  for  a  subset  of  consistent  models  to  be  selected  for 
further  processing.  A  total  of  three  model  changes  are  used  to  define  this  CMMA  application. 

The  models  are  a  change  in  propagation  path,  a  change  in  base  frequency,  and  a  change  in  the 
contact  velocity  vector  (maneuver).  The  three  model  changes,  along  with  the  original  model  (null 
hypothesis)  and  a  category  for  an  unclassified  model  anomaly,  yield  a  total  of  five  model 
hypotheses  from  which  a  selection  must  be  made. 
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Each  of  the  predicted  residual  sequences  of  the  five  measurements  that  are  available  from  the 
two  sensors  are  examined  for  anomalous  features  such  as  a  jump  in  the  data  sequence,  a  drift,  or 
both.  The  presence  of  these  features  in  the  predicted  residual  sequences  are  evidence  that  there 
has  been  a  change  in  the  model.  For  example,  a  change  in  the  propagation  path  from  direct  path 
to  bottom  bounce  results  in  a  jump  in  the  depression  and  conical  angles,  while  having  no  effect  on 
the  azimuthal  bearing.  The  features  are  subsequently  evaluated  by  the  evidential  reasoning  system 
to  determine  which  model(s)  could  (or  could  not)  have  produced  the  observed  effects.  The 
CMMA  gallery  that  is  representative  of  this  reasoning  process  is  discussed  in  greater  detail  in  the 
following  section. 


3.2  CMMA  GALLERY  DESCRIPTION 

An  overview  of  the  CMMA  evidential  reasoning  system,  or  gallery,  is  shown  in  figure  5.  In 
the  first  stage  of  the  CMMA  system,  evidence  is  extracted  from  the  available  measurements.  In 
this  case,  an  existing  state  estimate  and  the  underlying  process  model  are  used  to  obtain  predicted 
residuals.  The  predicted  residuals  are  examined  for  the  deterministic  features  of  jump  and  drift 
which  are  used  to  establish  the  basic  probability  assignments,  miA^,  in  the  feature  frame  of 
discernment.  One  feature  frame  of  discernment  is  established  for  each  measurement  type.  A 
compatibility  relation  is  used  to  map  the  BPA  in  the  features  for  each  measurement  type  into  BPA 
in  the  elements  of  the  model  hypotheses  frame  of  discernment.  Again,  there  is  one  compatibility 
relation  for  each  measurement  type.  Dempster's  combination  rule  is  used  to  combine  the  model 
hypotheses  frames  for  all  measurement  types  into  a  single  solution  frame  of  discernment.  The 
selection  criteria  can  then  be  applied  to  the  solution  frame  to  determine  which  modeling 
hypotheses  are  likely  to  be  consistent  with  the  evidence  presented  in  the  predicted  residuals  and, 
therefore,  should  be  considered  for  further  processing. 
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Figure  5.  Block  Diagram  of  CMMA  System 
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A  more  detailed  representation  of  the  first  stage  in  the  CMMA  gallery  is  given  in  figure  6. 
Here,  the  establishment  of  the  feature  frame  has  been  broken  down  into  its  constituent  parts  to 
facilitate  discussion. 


JUMP  FEATURE 
FRAME 


JOINT  FEATURE 
FRAME 


Figure  6.  Complete  Feature  Frame  Block  Diagram 


The  gallery  used  to  establish  the  feature  frame  of  discernment  is  divided  into  two  lines  or 
paths  of  reasoning.  The  first  line  of  reasoning  involves  evidence  pertaining  to  the  existence  of  the 
features  and  is  depicted  in  the  upper  half  of  figure  6.  The  second  line  of  reasoning  involves 
evidence  of  the  features'  amplitudes  and  is  shown  in  the  lower  half  of  figure  6.  Since  the  initial 
frames  of  discernment  established  for  each  type  of  evidence  are  different  but  related,  the 
intervening  stages  of  the  gallery  map  belief  from  the  initial  frames  into  compatible  frames  of 
discernment,  which  are  then  combined  to  produce  the  final  or  complete  frame  of  discernment. 
This  final  feature  frame  of  discernment  is  shown  as  the  right  most  block  in  figure  6  and  is  the 
feature  frame  used  in  figure  5. 

The  first  stage  of  the  gallery  comprises  two  feature  existence  frames  and  two  feature 
amplitudes  frames  of  discernment.  The  individual  frames  of  discernment  for  the  existence  of  a 
feature  are  shown  for  the  features  of  jump,  ©y  =  {jump  (j),  no-jump  (j)},  and  drift,  0^  =  {drift 

(d),  no-drift  (cf )},  in  the  upper  portion  figure  6.  For  each  measurement  type,  a  traditional 
likelihood  ratio  test  is  used  to  generate  evidence  pertaining  to  the  existence  of  the  features  of  a 
step  (jump)  and/or  a  ramp  (drift).  The  functions  that  map  the  evidence  of  a  feature  into  BPA  in 
the  feature's  existence  in  the  frame  of  discernment  are  discussed  in  section  3.3. 
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The  second  stage  in  establishing  the  feature  existence  are  the  joint  feature  frames  of 
discernment  qjd  =  {no  features  (]  d),  jump-only  (jd ),  drift-only  (y  d),  jump  and  drift  (jd)},  again 

one  for  each  measurement  type.  BPA  in  the  joint  feature  frame  is  obtained  from  BPA  in  frames  qj 
and  qj  through  the  compatibility  relationship  defined  in  table  4. 


Table  4.  Single  Feature  Frame  To  Joint  Feature  Frame 
Compatibility  Relationship 


Single 

Feature  Frame 

Joint  Feature  Frame 

{]} 

{]  djd} 

0. 

U) 

Udjd) 

{]J} 

{Jdjdjdjd} 

{d} 

{Jdjd} 

0. 

{d} 

{Jdjd} 

{d,d} 

{jdjdjd  Jd) 

Using  the  compatibility  map  of  table  4,  two  joint  feature  frames  are  generated,  one  for  the  jump 
feature  and  one  for  the  drift  feature.  From  this  table,  it  is  seen  that  the  existence  of  a  jump  feature 
in  9y  is  compatible  with  the  hypotheses  of  jump-only  or  jump  and  drift  (jd  Jd)  in  power  set  of  Qjd. 
Consequently,  the  BPA  for  jump  from  6y  is  mapped  into  BPA  for  the  element  (jd  Jd)  of  Gyj. 
Similar  reasoning  applies  to  the  remaining  terms  in  the  compatibility  relationship.  Note  that  BPA 
for  the  elements  of  the  individual  feature  frames  are  mapped  to  BPA  in  subsets  of  the  joint  feature 
frame,  so  BPA  can  be  assigned  directly  into  this  frame.  Although  the  system  BPA's  were 
implemented  directly  in  the  joint  feature  frame,  the  above  description  is  useful  in  developing  the 
evidence  representations. 

BP  As  in  the  two  joint  feature  frames  are  merged  using  Dempster's  combination  rule  to  form  a 
single  composite  feature  frame.  Note  that  although  the  constituent  joint  feature  frames  have  no 
BPA  in  the  singleton  hypotheses,  eg.,  { y  }  from  G^and  {d}  from  G^  map  and  combine  to  {jd}  in 

Qjd.  The  combined  frame  may  have  non-zero  BPA  for  the  singleton  hypotheses.  The  composite 
feature  frame  of  discernment  is  then  refined  to  reflect  amplitudes  in  the  feature  frame  of 
discernment  of  weak,  moderate,  and  strong.  The  feature  frame,  0/is; 

®f  =  {jdjdw,  jdm,  jds,  Jwdw ,  •  •  • ,  Jsdm,  jsds} , 

where  Jwdw  is  weak  jump  and  weak  drift,  etc.  The  refinement  is  implemented  with  the 
compatibility  map  defined  in  table  5.  Here,  for  example,  the  BPA  in  a  jump-only  is  compatible 
with  the  set  of  weak-jump  and  no  drift,  moderate-jump  and  no  drift  or  strong-jump  and  no  drift. 
The  other  relationships  are  similarly  defined  in  table  5. 
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Table  5.  Composite  Feature  Frame  to  Feature  Frame  Compatibility  Relationship 


0/. 

0/ 

{Jd) 

{Jd} 

[jd] 

{jwd  JmdJsd  } 

{Jd} 

{jdwj  dm,]  ds} 

{jd} 

{jwdwjmdw,jsdw,jwdm, . .  .jsds} 

The  second  line  of  reasoning  in  the  gallery  for  establishing  the  feature  frame  of  discernment  is 
the  feature  amplitude  frame.  Here,  the  estimates  of  the  jump  or  drift  amplitude  and  their  standard 
deviation,  obtained  from  equation  (22)  in  section  3.3,  are  used  to  establish  BP  A  in  the  amplitude 
of  the  feature  in  the  feature  frame.  The  functions  used  to  map  the  evidence  into  BPA  are 
discussed  in  section  3.4.  The  frame  of  discernment  is  {weak  feature,  moderate  feature,  and  strong 
feature).  There  are  separate  frames  for  both  the  jump,  0;^,  and  drift,  ©da,  features.  A 
compatibility  map  relates  the  BPA  in  feature  amplitude  to  BPA  in  the  feature  frame  and  is  given  in 
table  6  for  the  jump  feature.  A  similar  compatibility  relationship  is  used  for  the  drift  feature  and 
the  two  frames  are  combined  using  Dempster's  combination  rule.  Finally,  the  last  step  is 
combining  the  belief  in  the  feature  frame  established  by  the  evidence  for  feature  existence  with 
that  from  feature  amplitude  to  produce  the  final  feature  frame  shown  as  the  right  most  block  of 
figure  6  and  the  second  block  of  figure  5. 


Table  6.  Jump  Amplitude  to  Feature  Frame  Compatibility  Relationship 


Jump  Amplitude  Frame 

Feature  Frame  of  Discernment 

{/■s} 

[jsdjsdwjsdmjsds^  = 

{jm} 

1  jmd ,  jmdwjmdm,  jmd^  =  .<42 

{jw} 

{^jwd,jwdw,jwdm,jwds^  = 

{jmjs} 

A,{jA, 

{j^js} 

A,[jA, 

{Jwjm} 

{jwjmjs}  =  0^^ 

Referring  to  the  overall  CMMA  block  diagram  of  figure  5  again,  the  feature  frames  for  each 
of  the  residual  types  are  mapped  into  a  model  hypothesis  frame  using  a  compatibility  map.  The 
model  hypotheses  frame  of  discernment  comprises  the  model  events:  no  change  (Ho),  change  in 
base  frequency  (BF),  change  in  propagation  path  (PP),  contact  maneuver  (CM),  and  unknown 
anomaly  (NL).  There  is  one  compatibility  map  and  resultant  model  hypothesis  frame  for  each 
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residual  type.  The  compatibility  map  represents  the  logical  relationship  between  the  occurrence  of 
a  feature  set  in  a  residual  sequence  and  the  corresponding  model  hypotheses  that  could  be  the 
cause.  For  example,  a  change  in  propagation  path  results  in  a  weak,  moderate,  or  strong  jump  in 
the  depression  angle  with  a  respective  drift  amplitude  of  no  drift,  weak,  or  moderate.  Thus,  the 
events  {jwd  },{jmdw),{jsdm}  in  the  depression  angle  are  all  compatible  with  a  change  in  the 
propagation  path.  Consequently,  the  belief  in  all  the  subsets  of  {jwd  JmdwJsdm)  in  DE  are 
compatible  with  the  model  hypothesis  of  change  in  propagation  path.  Thus,  their  BP  As  are 
assigned  to  the  model  hypothesis  of  change  in  propagation  path.  If  a  subset  of  the  feature  frame 
is  compatible  with  more  than  one  model  hypothesis,  its  belief  is  associated  with  the  union  of  the 
model  hypotheses.  The  compatibility  relationships  were  determined  using  a  perturbation  analysis, 
and  were  verified  experimentally  via  scatter  diagrams.  There  are  over  90  relationships  in  the 
compatibility  map.  The  compatibility  relationships  are  detailed  in  appendix  A  and  the  perturbation 
analysis  is  detailed  in  appendix  E. 

The  final  solution  is  obtained  by  combining  all  the  beliefs  in  all  the  model  hypothesis  frames 
using  Dempster's  combination  rule.  At  this  point,  the  selection  criterion  can  be  applied  to  select 
modeling  hypotheses  for  further  processing  of  the  data  as  indicated  in  figure  2.  Although  this 
CMMA  application  considered  limited  amounts  of  evidence  and  model  hypotheses,  the  framework 
of  this  gallery  is  readily  extendible  to  include  additional  evidence  from  features  such  as  curvature 
in  the  predicted  residual  sequence,  additional  evidence  obtained  directly  from  the  measurements 
or  other  sources,  as  well  as  additional  model  hypotheses. 


3.3  FEATURE  EXISTENCE  EVIDENCE  REPRESENTATIONS 

The  evidence  representations  map  the  evidence  pertaining  to  the  existence  of  features  in  the 
predicted  residuals  into  belief  in  the  feature  frame  of  discernment.  Although  these  are  not 
evidence  representations  as  defined  by  Shafer,'^  the  usage  of  that  term  has  pervaded  many  of  the 
references  and  is,  therefore,  retained  here.  Note,  however,  that  the  correct  term  is  "a  function 
that  assigns  basic  probability." 

As  mentioned  in  the  preceding  section,  the  belief  in  the  feature  frame  is  established  in  two 
stages.  In  the  first  stage,  belief  in  the  existence  of  the  feature  is  established  using  a  test  statistic 
from  a  multiple  hypothesis  test.  The  test  statistic  is  an  estimate  of  the  signal  to  noise  ratio  of  the 
various  features.  It  is  computed  as  the  squared  difference  between  two  regression  fits  to  the 
residuals,  one  with  all  the  coefficients  (features),  the  other  with  some  coefficients  set  to  zero.^^ 
The  belief  in  a  feature  is  based  on  the  difference  between  the  probability  of  detection  (PD)  and 
probability  of  false  alarm  (PF),  with  the  threshold  set  at  the  point  of  equal  likelihood  of  the  two 
densities  associated  with  PD  and  PF.  The  belief  in  no  feature  is  established  as  the  joint  probability 
of  two  (assumed)  independent  events:  the  probability  of  false  alarm,  and  the  belief  in  the  ability  to 
detect  a  reference  level  feature  in  the  given  noise  level.  The  approach  of  the  first  stage  is 
summarized  in  this  section  and  the  details  are  given  in  appendix  C  and  by  Ferkinhoff,  et  al.*^  The 
second  stage  is  a  refinement  of  the  belief  in  a  feature  into  belief  in  a  particular  amplitude  for  the 
feature,  that  is ,  weak,  moderate,  or  strong  jump  and/or  drift.  These  beliefs  are  established  by 
computing  the  expected  value  of  the  membership  functions  for  each  of  the  amplitudes:  weak, 
moderate,  or  strong.  The  second  stage  is  detailed  in  Appendix  D  and  by  Ferkinhoff.’^ 
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3.3.1  EVIDENCE  EXTRACTION 


Evidence  of  features  in  the  predicted  residual  sequence  is  obtained  in  the  form  of  the 
estimated  coefficients  of  a  polynomial  regression  fit.  Only  the  pertinent  results  are  presented  in 
this  section.  Additional  detail  may  be  found  in  appendix  B.  Consider  the  following  discrete  time 
dynamic  state  model  that  generates  measurements,  zik)\ 


x{k  +  1)  =  +Bu{k)  +  rM'(A:), 


(17a) 


z{k)=H[x{k)]  +  n{k). 


(17b) 


Here,  xijc)  is  the  nxl -dimensional  state  vector,  u  is  the  input  vector,  w  is  zero-mean,  white, 
Gaussian  process  noise,  z  is  the  measurement,  and  n  is  zero-mean,  white,  Gaussian  measurement 
noise,  which  is  independent  of  the  process  noise.  The  matrices  <t>,  B,  T  are,  respectively,  the 
state  transition  matrix,  input  matrix,  and  process  noise  input  matrix.  The  function  //[*]  is  the 
nonlinear  function  relating  the  state  to  the  measurement  and  k  is  the  time  index.  The  state  jr(A:)  is 

available  only  as  an  estimate,  x(k\k),  along  with  the  estimated  error  covariance  matrix,  P(k\k).  The 

estimate  x(^|^)  of  the  state  at  time  k  is  based  on  all  the  past  data  up  to  and  including  time  k.  The 
estimate  is  available  fi'om  a  suitable  estimator  such  as  an  extended  Kalman  filter  or  maximum 

likelihood  estimator.  The  predicted  residual  z{k\k-\)  is  the  difference  between  the  measurement 

z{k)  and  the  estimate  of  the  measurement,  z(A:|  A:- 1),  that  is. 


z(Jc\k-\)  =  z(k) -z{k\k-\). 


(18a) 


where 


z{k\k-\)  =  H[x(k\k-\)l 


(18b) 


x(A:i;t-l)  =  i^(^-llyt-l)  +  Bu{k-\), 


(18c) 


and  wherex(^|A:-l)  is  the  state  estimate  at  time  k  based  on  the  measurements  up  to  and  including 
time  k-\  but  not  time  k.  The  process  to  be  investigated  here  is  the  predicted  residual  sequence. 


Z  -  [z\,  Z2,  .  .Zj^] 


(19a) 


where 


Zj  =  z(k  +  i\k). 


(19b) 
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(19c) 


z{k  +  i\k)  =  z{k  +  i)-H  [x(it  +  /|)t)], 
x{k  +  /|^)]  =  Px{k\k)  +  ^  Bu{k  +  >1).  (19d) 


Here,  as  well  as  in  appendix  B,  only  a  single  measurement  type  is  considered  at  a  time.  The 
predicted  residual  sequence  can  be  approximated  as  a  jointly  Gaussian  random  vector.  However, 
the  approximation  only  remains  valid  whenever  the  second-order  and  higher  terms  of  the  Taylor 
series  expansion  of  the  measurement  nonlinearity  can  be  ignored.  Thus,  two  assumptions  on  the 
predicted  residual  sequence  are  invoked  to  assure  the  reasonableness  of  the  Taylor  series 
approximation.  First,  the  dynamic  model  of  equation  (17),  which  is  used  to  obtain  the  estimate, 
must  be  a  reasonably  accurate  description  of  the  real  process.  Second,  the  state  error,  and  its 
effect  over  the  time  interval  of  interest,  must  remain  sufficiently  small  to  allow  the  validity  of  a 
first-order  Taylor  series  approximation  of  the  predicted  residual  sequence.  These  two 
assumptions  are  the  basis  of  the  null  hypothesis  (//o  ’  no  modeling  error  and  a  good  state 
estimate). 


Under  the  Ha  hypothesis,  the  predicted  residual  process  is  a  zero-mean  jointly  Gaussian 
random  vector;  that  is,  the  probability  density  function  is 


P(Z|//0)  =  [7^"'  det  I V '  exp 


2 


'~z 


(20a) 


where  V  is  the  covariance  matrix  given  by, 

f  =  f[zz’’]-/:[z]£[z^]. 


Note  that  although  elements  of  the  predicted  residual  vector  are  correlated,  there  always  exists  a 

set  of  basis  vectors  in  which  Z  can  be  represented  by  uncorrelated  (and  hence  independent) 
components.  For  the  applications  considered  here,  both  process  noise  and  state  estimation  error 
covariances  are  small  compared  with  the  measurement  noise  covariance.  For  these  cases  the 

elements  of  Z  are  also  independent 

Under  the  alternate  hypotheses,  HjJ  ^  0,  there  is  a  modeling  error.  The  original  state  and 
measurement  models  of  equations  (17)  do  not  adequately  describe  the  system  dynamics.  The 
modeling  errors  may  arise  from  unknown  system  inputs,  a  change  in  the  measurement  model,  or 
an  unobserved  change  in  the  measurement  noise  statistics.  Examples  of  such  modeling  anomalies 
are,  respectively,  a  target  maneuver,  change  in  propagation  path,  and  measurement  bias. 
Regardless  of  the  cause  of  the  modeling  error,  its  observable  effect  is  often  a  feature  in  the 
predicted  residual  sequence. 

Under  the  mismodeling  hypotheses  HjJ  ^  0,  the  predicted  residuals  exhibit  features  that  are 
distinguishable  from  those  of  the  hypothesis  Ha.  For  the  hypothesis  Hj,j  l)-the  order 
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polynomial  is  used  to  model  the  anomalous  features  of  the  predicted  residuals.  The  probability 
density  function  for  the  jointly  Gaussian  random  vector  under  HjJ  ^  0,  is 

F(Zl//y)  =  i/er  irr  '  exp  ,7^0  (21a) 

where, 

Mi=Bm,  (21b) 


1  0  0 
1  (^1  -  ^o)  (^I  '  ^o)  •  •  • 


and 

/Wo 

/w, 

1.  (21d) 


The  features  of  interest  are  extracted  from  a  regression  fit  of  a  second-order  polynomial  to  the 
residuals  referenced  to  time  to: 

m^[B^BYB^[Z-Z,{x)],  (22) 

where  Zo(x)  are  the  predicted  measurements  associated  with  hypothesis  Ho,  the  B  matrix  is  given 

in  equation  (21c),  and  m  =  [wo,w,,W2]^  is  the  vector  of  estimated  regression  coefficients 

identified  with  the  specific  features  of  jump  (mo),  drift  (mi),  and  curvature  (m2).  Although 
curvature  and  higher  order  terms  may  be  present  in  the  residual  sequence,  they  are  not  used 
presently  as  evidence,  however,  their  presence  may  be  necessary  in  the  regression  fit  to  obtain 
unbiased  estimates  of  the  jump  and  drift  coefficients.  Here,  the  significance  of  the  curvature 
coefficient  is  rejected  if  its  test  statistic  (discussed  below)  falls  below  a  threshold  set  to  achieve  a 
10-percent  false  alarm  rate.  Other  techniques  are  also  available  to  determine  the  appropriate 
order  of  the  regression  fit.'* 


(21c) 
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An  example  of  a  noise-free  residual  plot  is  shown  in  figure  7.  Here,  Pa  Pc  have  been 
displayed.  The  figure  displays  the  difference  between  the  measurement  and  its  estimate  on  the 
horizontal  axis  with  time  progressing  downward  on  the  vertical  axis.  For  the  purposes  of 
illustration,  there  is  essentially  no  noise  on  the  measurements.  Data  up  to  and  including  the  time 
line  marker  labeled  tr  have  been  used  to  obtain  the  state  estimate.  Data  subsequent  to  the  time  line 
are  the  predicted  residuals.  Here,  the  process  changed  from  direct  path  propagation  to  bottom 
bounce.  The  predicted  residual  plot  for  conical  angle  exhibits  a  jump  and  a  slight  drift;  the 
azimuthal  bearing  exhibits  no  features.  The  following  sections  explain  how  this  evidence  may  be 
used  to  limit  the  class  of  models  to  those  that  can  produce  these  effects. 


CONICAL  ANGLE 


AZMUTHAL  BEARING 


TIME  LINE  t 


Figure  7.  Noise-Free  Residuals  Plot 


3.3.2  EVroENCE  REPRESENTATION 

The  basis  for  distributing  belief  are  the  feature  detection  and  false  alarm  probabilities,  Pp  and 
Pf  obtained  from  a  multiple  hypothesis  likelihood  ratio  test.  The  test  statistic  /  is  compared  with  a 
threshold  e  for  the  multiple  hypothesis  test  as 

H 

li  ‘e,  (23) 

Hj 


where  Hj  is  the  hypothesis  that  all  features  exist.  Hi  is  the  hypothesis  that  the  /th  feature  is  not 
significant,  and 


/  = 


M-M. 


(24) 


22 


Here,  =  Bm^ ,  where  the  are  separate  estimates  with  the  appropriate  coefficients  set  to 
zero,  and  the  test  statistic  /  is  an  estimate  of  the  signal(feature)-to-noise  ratio  (SNR). 
Traditionally,  the  threshold  is  set  to  satisfy  a  specified  probability  of  false  alarm  criterion. 

The  test  statistic  is  the  squared  difference  between  the  two  regression  fits,  Mj  with  all  the 

coefficients  and  with  some  coefficients  set  to  zero.  With  Gaussian  distributed  errors  and 
known  variance  the  test  statistic  is  a  chi-squared  random  variable  with  the  number  of  degrees  of 
freedom  equal  to  the  number  of  coefficients  being  tested,  one  in  this  case.  Since  the  test  statistic 
is  chi-squared  with  one  degree  of  freedom,  it  may  be  alternatively  viewed  as  the  square  of  a 
univariant  Gaussian  random  variable.'®  Under  the  hypothesis  that  a  feature  is  not  present,  the 
corresponding  Gaussian  distribution  is  zero  mean.  Under  the  hypothesis  that  a  feature  is  present, 
the  Gaussian  distribution  has  a  non-zero  mean  equal  to  the  square  root  of  the  test  statistic.  In  the 
absence  of  any  prior  knowledge  about  the  feature,  the  threshold  is  taken  at  the  point  of  equal 
likelihood  of  the  two  densities,  which  is  at  e  =  -Jl  /2.  Note  that  the  threshold  varies  with  the 
estimated  SNR  and,  therefore,  responds  dynamically  to  changing  signal  or  noise  levels.  While  the 
assumption  of  known  variance  is  used  here,  unknown  variance  can  also  be  handled  by  substituting 
an  F-test  for  the  chi-squared  test. 

The  Pf  is  associated  with  the  belief  that  the  estimated  SNR  value  belongs  to  the  "no  feature" 
distribution  (true  SNR  =  0),  while  Pd  is  associated  with  the  belief  that  the  estimated  SNR  value 
belongs  to  the  "feature  present"  distribution  (true  SNR  #  0).  The  belief  that  a  feature  is  present 
is  directly  proportional  to  the  estimated  SNR,  which  is  analogous  to  a  signal  excess  (excess 
beyond  SNR  =  0).  Consequently,  the  belief  that  a  feature,  i.e.  jump,  is  present  is  defined  as  the 
difference  between  Pd  and  Pp-' 

mQ)  =  PD  -  Pf-  (25) 

The  belief  in  no  feature  (no-jump)  more  complicated.  Two  separate  conditions  lead 

to  zero  SNR.  In  one  case,  noise  may  approach  infinity;  in  the  other,  the  signal  level  may 
approach  zero.  In  the  large  noise  case,  although  SNR  is  approaching  zero,  the  placing  of  all 
belief  into  the  no  feature  hypothesis  results  in  low  plausibility  of  a  feature  being  present.  This  is 
clearly  undesirable,  because  although  high-noise  inhibits  a  feature  detection,  one  could  be 
present;  thus  the  plausibility  of  the  existence  of  a  feature  should  be  high.  For  the  simple  structure 
of  the  individual  feature  frames,  a  high  plausibility  for  the  feature  implies  a  low  belief  in  no 
feature.  Thus,  /»( j )  should  be  a  function  of  both  the  estimated  SNR  as  well  as  a  measure  of  the 
significance  of  the  noise  level.  Following  this  line  of  reasoning,  m(j)  is  established  as  the  joint 
probability  of  two  (assumed)  independent  events:  the  probability  of  false  alarm  Pp- given  the 
estimated  SNR,  and  the  belief  in  the  ability  to  detect  a  reference  level  feature  in  the  given  noise 
level.  Consequently 


w(7  )  —  {Pdr .  Pfr)  Pf, 


(26) 


where  Pdr  and  Pfr  are,  respectively,  the  detection  and  false  alarm  probabilities  of  a  reference 
signal  in  the  given  noise  level.  The  reference  signals  can  be  set  at  the  minimum  level  of  bias  that 
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can  be  reasonably  tolerated.  The  importance  of  the  reference  signal  level  is  discussed  in  the 
experimental  results  section.  Because m{j)< Pp,  it  follows  that 

in{j)  +  m(j)<PD<\,  (27) 

and,  since  the  BP  As  must  sum  to  unity,  the  remaining  belief  is  assigned  to  the  frame  and 
represents  ignorance  as 

m((dj)  =1-Pd  +  [1-(Pdr-  PfrWf  .  (28) 

A  more  detailed  discussion  of  the  functions  that  map  the  evidence  of  features  in  the  predicted 
residuals  into  the  basic  probability  assignments  of  the  feature  existence  frames  is  given  by 
FerldnhofF,  et  al.’”’^^ 


3,4  FEATURE  AMPLITUDE  EVIDENCE  REPRESENTATIONS 

The  second  line  of  reasoning  involved  evidence  of  the  features'  amplitude,  that  is,  weak, 
moderate,  or  strong  jump  or  drift.  Evidence  of  feature  amplitude  is  embedded  in  the  probability 
density  fiinction  of  the  estimated  features,  or  regression  coefficients  of  equation  22.  The  feature 
density  function  is  approximated  with  a  Gaussian  density  function  where  the  estimated  features 
are  used  as  the  mean  values  and  the  covariance  is  obtained  from  equation  22.  The  functions  that 
map  evidence  of  weak,  moderate,  and  strong  features  into  the  basic  probability  assignments  in  the 
feature  amplitude  frame  of  discernment  are  illustrated  graphically  below  in  figures  8a  through  8c 
and  are  discussed  in  greater  detail  in  appendix  D  and  by  FerkinhofF,  et  al.’"* 

As  an  example,  the  probability  density  function  for  a  feature,  jump,  is  shown  in  figure  8a. 
Figure  8b  illustrates  the  regions  defined  as  weak,  moderate,  and  strong  jumps  with  polarity 
discrimination  (±).  Note  that  in  this  construction,  the  boundaries  of  the  membership  functions 
can  overlap,  thus  allowing  for  a  fiizzy  set  interpretation  for  feature  amplitudes.  The  expected 
values,  for  each  jump  amplitude;  strong  negative  (-js),  moderate  negative  (-jnt),  weak  negative  or 
positive  (Jw),  moderate  positive  (+/w),  and  strong  positive  {+js),  are  obtained  as  the  integration  of 
the  product  of  the  feature's  probability  density  and  membership  functions,  and  are  illustrated  in  the 
histogram  of  figure  8c.  The  BPA  is  assigned  from  this  histogram.  Level  cuts  are  made  at  each 
value  as  shown  in  the  figure  with  dashed  lines.  These  level  cuts  are  projected  to  the  scale  on  the 
far  right.  The  BPA  is  the  difference  in  these  values  times  the  number  of  sets  intersected  in  the 
histogram.  The  resulting  BPA  value  is  assigned  to  the  union  of  the  sets  that  have  been  intersected 
by  the  area  between  dashed  lines  in  the  plot.  However,  because  the  amplitude  frame  does  not 
currently  distinguish  between  positive  and  negative  features,  the  final  BPA  assignments  are 
without  regard  to  polarity.  The  compatibility  relation  between  frames  with  and  without  polarity 
are  shown  in  table  7. 
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Figure  So.  Feature  Amplitude  Probability  Density  Function 
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Figure  8b.  Feature  Amplitude  Membership  Functions 


Figure  8c.  Feature  Amplitude  Expected  Value  and  BPA 
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Table  7.  Compatibility  Relation  Between  Feature  Amplitudes  With  and  Without  Polarity 


Feature  Amplitude  Frame  With  Feature  Amplitude  Frame  Without 
Polarity  Polarity 


Note  that  for  single  mode  probability  density  functions,  like  the  Gaussian,  logically 
inconsistent  assignments  of  BPA  to  sets  like  {-^5, +75)  or  {jw,-js}  cannot  occur  with  this 
technique.  Consequently,  non-zero  BPA  for  both  sets  {-js}  and  {+js}  cannot  occur 
simultaneously,  however,  a  non-zero  BPA  assignment  for  the  sets  {-js}  and  {-Js,-jm,jw,+Jm,+js} 
is  possible  and  does  not  create  a  logically  inconsistent  result.  Also,  because  the  membership 
functions  that  define  the  regions  of  feature  amplitudes  sum  to  unity  for  all  values  of  jump 
amplitude  in  figure  8b,  a  total  BPA  of  unity  is  obtained  in  the  frame  of  discernment. 

The  boundaries  for  the  feature  amplitude  regions  were  obtained  from  a  computer  simulated 
perturbation  analysis.  The  perturbation  analysis  was  performed  numerically  with  a  monte  carlo 
simulation  and  the  results  presented  in  the  form  of  scatter  plots  of  event  occurrence  in  a  jump- 
drift  feature  plane.  The  scatter  plots  are  discussed  in  appendix  E.  The  boundaries  were 
established  by  visual  inspection;  no  attempt  was  made  to  optimize  the  boundary  locations. 


4.  EXPERIMENTAL  RESULTS 


Experimental  results  were  obtained  using  synthetic  data  generated  by  a  computer  simulation. 
As  described  in  the  previous  section,  the  CMMA  simulation  involved  a  single  contact  and  a  single 
observation  platform  with  two  spatially  diverse  sensors.  The  available  measurements  are: 
azimuthal  bearing,  depression  angle,  and  frequency  from  the  first  sensor,  and  conical  angle  and 
frequency  from  the  second  sensor.  The  measurement  noise  standard  deviation  were  arbitrarily 
selected  as  0.3°,  0.5°,  0.03,  0.7°,  and  0.04  Hz,  respectively.  The  data  rate  for  all  measurements 
are  20  seconds. 

Initially,  the  contact  is  located  5000  m  from  the  observer  at  a  bearing  of  10°  and  moves  on  a 
course  of  330°  with  a  speed  of  12  m/s.  The  base  frequency  is  a  nominal  800  Hz.  The  observer 
executes  a  three-leg  geometry,  where  each  leg  is  a  segment  of  constant  velocity.  For  these 
experiments,  the  time  on  each  leg  is  900  s  and  the  course  and  speed  for  the  legs  are  300°  and  14 
m/s,  60°  and  14  m/s,  and  340°  and  12  m/s,  respectively. 

The  data  from  the  first  two  ownship  legs  are  processed  using  the  correct  model  hypotheses. 
The  segment  of  data  from  the  third  leg  is  examined  for  model  consistency  and  the  evidential 
reasoning  process  is  used  to  select  a  subset  of  model  hypotheses  for  further  processing.  For  the 
purposes  of  these  experiments,  there  is  only  a  single  change  in  the  model  which  is  known  to  occur 
in  the  middle  of  the  third  leg.  The  set  of  model  changes  is  a  change  in  propagation  path  from 
direct  path  to  bottom/surface  reflection  (PP),  a  contact  maneuver  (CM),  a  change  in  the  base 
frequency  (BF),  no  change  in  the  model  or  the  null  hypothesis  (HO),  and  an  unknown  model 
change  (NL). 

The  experimental  results  are  contained  in  table  8.  Each  experiment  is  represented  by  the 
entries  in  a  column.  The  rows  of  the  table  are  the  set  of  hypotheses  selected  by  the  CMMA 
system  over  all  the  experiments.  The  entries  in  the  table  are  the  percent  of  time  that  the 
corresponding  hypotheses  set  for  the  given  experiment  were  selected.  The  selection  threshold 
was  set  at  the  support  level  of  0.9.  For  each  experiment  the  results  were  obtained  from  100 
monte  carlo  trials.  Six  different  experiments  were  conducted,  involving  five  different  modeling 
anomalies  and  one  that  had  no  model  change.  The  first  experiment,  shown  in  column  one,  was  a 
small  contact  maneuver  (SMCM)  involving  a  course  change  of  5°  to  a  new  heading  of  335°.  The 
second  experiment  was  a  larger  contact  maneuver  (LGCM)  of  15°  to  a  new  course  of  345°.  The 
third  experiment  was  a  small  change  in  the  propagation  path  (SMPP)  from  direct  path  to  bottom 
bounce  with  a  shallow  ocean  bottom  depth  of  500  m.  The  fourth  experiment  was  a  large  change 
in  propagation  path  from  direct  path  to  bottom  bounce  with  a  bottom  depth  of  3000  m.  The  fifth 
experiment  was  a  change  in  the  base  frequency  (BF)  of  1 5  Hz.  The  last  experiment  had  no  model 
error  and  is  the  null  hypothesis  (HO). 

As  shown  by  the  results  in  table  8  for  the  SMCM  experiment,  the  hypothesis  set  of  change  in 
base  frequency,  propagation  path,  or  contact  maneuver  was  selected  most  often  (52  percent) 
while  the  sets  base  frequency  or  contact  maneuver,  and  propagation  path  or  contact  maneuver 
were  selected  25  percent  and  22  percent,  of  the  time  respectively.  The  incorrect  hypothesis  base 
frequency  was  selected  1  percent  of  the  time  while  contact  maneuver  alone  was  never  selected. 
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Table  8.  Experimental  Results 


SMCM 

LGCM 

SMPP 

LGPP 

BF 

HO 

{H0,BF,PP,CM) 

0 

0 

1 

0 

0 

22 

{BF,PP,CM} 

52 

0 

1 

0 

0 

1 

{H0,BF,PP] 

0 

0 

0 

0 

0 

14 

{H0,PP,CM\ 

0 

0 

27 

0 

0 

14 

{BF,CM) 

25 

0 

0 

0 

0 

0 

{PP,CM) 

22 

0 

70 

0 

0 

0 

{H0,PP] 

0 

0 

0 

0 

0 

1 

{H0,BF) 

0 

0 

0 

0 

0 

33 

{CM} 

0 

100 

1 

0 

0 

0 

{PP} 

0 

0 

0 

100 

0 

0 

[BF] 

1 

0 

0 

0 

100 

2 

_ m _ 

0 

0 

0 

0 

0 
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Since  this  contact  maneuver  is  small  and,  therefore,  of  limited  observable  consequence,  it  is  not 
surprising  that  the  correct  modeling  hypothesis,  contact  maneuver,  is  never  uniquely  identified. 
More  importantly,  however,  is  that  the  correct  hypothesis  is  included  99  percent  of  the  time. 

Thus,  for  this  example,  the  evidential  reasoning  is  not  very  decisive  but  is  inclusive.  Compare  the 
results  of  the  small  contact  maneuver  in  column  one  with  the  results  for  the  larger  contact 
maneuver  shown  in  column  two;  here,  the  correct  hypothesis  is  always  selected.  Clearly  the 
larger  contact  maneuver,  which  has  a  more  observable  effect  on  the  residuals,  leads  to  more 
decisive  reasoning. 

Similar  results  are  also  observed  for  the  small  and  large  changes  in  propagation  path  given  in 
columns  three  and  four.  Note  however,  that  for  small  changes  in  the  propagation  path,  although 
the  correct  hypothesis  is  included  99  percent  of  the  time,  it  is  associated  with  the  null  hypothesis 
28  percent  of  the  time  .  That  is,  27  percent  of  the  time  the  evidential  reasoning  process  concludes 
that  either  a  change  in  propagation  path,  contact  maneuver,  or  nothing  happened  and  1  percent  of 
the  time  concludes  {H0,BF,PP,CM}.  When  changes  in  the  model  are  associated  with  HO,  a 
conclusion  that  may  be  drawn  is  that  whatever  may  have  occurred  is  of  such  a  small  effect  as  to 
be  confused  with  HO.  Therefore,  one  approach  is  to  ignore  the  possible  model  changes  and  select 
HO  until  further  evidence  warrants  a  different  conclusion.  This  confusion  with  HO  is  apparent  in 
the  null  hypothesis  experimental  results  given  in  the  last  column.  Here  the  correct  hypothesis,  HO 
is  included  97  percent  of  the  time  while  the  false  detection  rate  is  3  percent.  Note  that  HO  alone  is 
selected  only  13  percent  of  the  time.  Currently,  it  is  believed  that  this  behavior  is  attributable  to 
the  manner  in  which  the  evidence  is  generated  and  represented  for  CMMA  and  is  being 
investigated  further. 

The  results  presented  are  typical  of  those  obtained  from  many  simulations  and  have  been 
selected  as  representative.  However,  extensive  analysis  under  conditions  of  poor  observability 
with  limited  data  were  not  attempted.  Thus,  although  current  results  indicate  that  performance 
degrades  gracefully  with  deteriorating  conditions,  a  complete  analysis  of  that  subject  has  not  been 
conducted. 
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5.  CONCLUSION 


The  application  of  DS  theory  of  evidential  reasoning  to  the  contact  management  model 
assessment  problem  was  examined.  This  report  is  a  collection  of  results  that  have  been  reported 
separately  elsewhere  but  are  presented  together  here  for  the  first  time.  Several  results  were 
obtained.  Evidence  representations,  that  map  evidence  from  statistical  hypothesis  test  to  the 
belief  in  the  existence  of  features  is  developed 

A  method  for  hypothesis  selection  in  the  DS  framework  of  evidential  reasoning  was 
developed.  The  method  involves  arranging  the  subsets  of  the  frame  in  order  of  decreasing  BPA 
values.  For  each  distinct  value  of  BPA  all  the  subsets  with  BPA  equal  to  or  exceeding  that  value 
are  unioned,  and  the  support  for  the  unioned  set  determined.  The  process  is  continued  with 
decreasing  BPA  values  until  a  threshold  on  the  support  is  met  or  exceeded.  The  resulting  set  is 
the  set  of  selected  hypotheses.  The  selection  method  was  applied  to  the  evidential  reasoning 
processes  of  the  contact  management  model  assessment  problem  and  experimental  results  were 
obtained  with  synthetic  data.  Although  limited  results  were  obtained,  the  selection  criterion 
achieved  acceptable  performance  in  that  the  selected  set  included  the  correct  model  hypothesis. 
The  selection  method  is  efficient  in  that,  at  most,  it  only  requires  the  computations  of  support  for 
the  elements  of  the  power  set  with  non-zero  BPA  and  their  unions.  A  geometric  representation 
for  the  frame  based  on  BPA  values  and  the  measures  of  support  and  plausibility  was  also 
presented  and  has  provided  insight  into  the  evidential  reasoning  process. 

Evidence  representations  were  developed  that  map  the  evidence  obtained  from  a  statistical 
hypothesis  test  to  belief  in  the  features  of  jump  and  drift.  The  belief  is  established  in  two  stages. 
In  the  first  stage,  belief  in  the  existence  of  the  feature  is  established  using  a  test  statistic  from  a 
multiple  hypothesis  test.  The  test  statistic  is  an  estimate  of  the  signal  to  noise  ratio  of  the  various 
features.  It  is  computed  as  the  squared  difference  between  two  regression  fits  to  the  residuals, 
one  with  all  the  coefficients  (features),  the  other  with  some  coefficients  set  to  zero.  The  belief  in 
a  feature  is  based  on  the  difference  between  the  probability  of  detection  (PD)  and  probability  of 
false  alarm  (PF),  with  the  threshold  set  at  the  point  of  equal  likelihood  of  the  two  densities 
associated  with  PD  and  PF.  The  belief  in  no  feature  is  established  as  the  joint  probability  of  two 
(assumed)  independent  events:  the  probability  of  false  alarm,  and  the  belief  in  the  ability  to 
detect  a  reference  level  feature  in  the  given  noise  level.  The  second  stage  is  a  refinement  of  the 
belief  in  a  feature  into  belief  in  a  particular  amplitude  for  the  feature,  that  is ,  weak,  moderate,  or 
strong  jump  or  drift  (or  both).  These  beliefs  are  established  by  computing  the  expected  value  of 
the  membership  functions  for  each  of  the  amplitudes.  Beliefs  from  both  stages  are  then 
combined  using  Dempster's  combination  rule. 

Experimental  results,  using  simulated  data,  were  presented  that  demonstrated  the  viability  of 
the  DS  theory  of  evidential  reasoning  to  the  contact  management  model  assessment  problem.  In 
almost  all  cases  (99  percent)  the  correct  model  hypothesis  was  included  in  the  selected 
hypothesis  set.  For  scenarios  with  strongly  observable  features  the  correct  hypothesis  was  almost 
always  the  single  hypothesis  selected  from  the  universal  set.  The  case  for  no  model  change 
presented  some  confused  results  in  that  it  was  seldom  the  single  selection,  however  it  almost 
always  included  in  the  selected  set.  The  cause  for  this  confusion  is  believed  to  be  the  result  of 
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not  having  a  membership  set,  in  the  feature  amplitude  frame  of  discernment,  to  indicate  that  the 
detected  feature  was  a  very  small  feature  of  no  consequence. 

Suggestions  for  future  work  are  to  expand  the  sources  of  evidence  to  include  the  processed 
residuals  under  the  selected  models,  evidence  in  the  measurements,  and  solution  quality  regions 
for  the  multiple  hypothesis  estimates.  Further  investigation  into  the  use  of  fuzzy  sets,  fuzzy  rule 
based  reasoning,^®  and  the  use  of  possibility  theory  are  recommended.  Additionally,  Bayesian 
inference  nets  may  also  be  investigated. 
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APPENDIX  A 

COMPATIBILITY  RELATIONS 


The  CMMA  compatibility  relationships  for  the  five  measurements  of  sphere  bearing,  sphere 
depression  angle,  sphere  and  towed  array  frequency,  and  conical  angle  are  given  in  tables  A-1 
through  A-4.  Due  to  the  large  frame  size  only,  the  compatibility  relations  between  the  singletons 
of  both  fi'ames  are  given.  Since  the  compatibility  of  a  union  equals  the  union  of  the  compatible 
subsets,  all  remaining  relationships  between  the  supersets  of  the  singletons  may  be  readily  derived. 
These  relationships  were  obtained  using  both  an  analytical  and  numerical  perturbation  analysis. 
The  results  of  the  numerical  analysis  are  contained  in  appendix  E. 

Referring  to  table  A-1,  the  feature  frame  has  16  elements, 

6f-  {]  d  jwd  Jmd  Jsd ,  j dw,] dm,] ds,jwdw,jwdm,jwdsjmdw,jmdmjmds,jsdw,jsdm,jsds} 

which  are  weak,  moderate,  and  strong  jump  with  no  drift  (jwd  jmd  jsd ),  respectively,  weak, 
moderate,  strong  drift  with  no  jump  ( J  dw,  j  dm,  j  ds),  respectively;  and  all  their  pairwise 
combinations.  The  elements  of  the  model  frame 

{HO,BF,PP,CM,NL} 

are:  no  model  change  (HO),  change  in  base  frequency  (BP),  change  in  propagation  path  (PP), 
contact  maneuver  (CM),  and  an  unidentifiable  model  change  (NL).  The  x's  entered  in  the  table 
signify  compatible  events.  For  the  sphere  bearing  compatibility  relations  given  in  table  A-1,  no 
feature  (J  d)is  compatible  with  no  model  change,  a  change  in  base  frequency,  or  a  change  in 
propagation  path,  thus  the  mapping  is 

{]d}-^{H0,BF,PP} 

A  similar  interpretation  can  be  given  to  the  remaining  entries  in  all  the  tables. 
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Table  A-l.  Compatibility  Relations  for  Sphere  Bearing 
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Table  A~2.  Compatibility  Relations  for  Sphere  Depression  Angle 
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pp 

CM 
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X 
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X 
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jmdw 

X 

jmdm 

X 
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X 
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X 
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X 
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Table  A-3.  Compatibility  Relations  for  Sphere  and  Towed  Array  Frequency 


A-4 


Table  A’4,  Compatibility  Relations  for  Towed  Array  Conical  Angle 
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BF 
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CM 

NL 
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X 

X 

jwd 

X 
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X 

X 

X 

X 

jmdw 

X 

jmdm 

X 

jmds 

X 
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X 
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Reverse  Blank 


APPENDIX  B 
PROCESS  MODEL 


Consider  the  following  discrete,  time-dynamic  state  model  that  generates  measurements,  z{ky. 


x(k+l)  =  Fx{k)  +  Bu{k)  +  Gw{k), 


(B-la) 


z(k)=H[x{k)]-^n{k). 


(B-lb) 


Here,  x(A:)  is  the  m\  dimensional  state  vector,  u  is  the  input  vector,  w  is  zero-mean,  white, 
Gaussian  measurement  noise,  z  is  the  measurement,  and  n  is  zero-mean,  white,  Gaussian 
measurement  noise,  which  is  independent  of  the  process  noise.  The  matrices  F,  B,  G  are, 
respectively,  the  state  transition  matrix,  input  matrix,  and  process  noise  input  matrix.  The 
function  H  is  the  nonlinear  function  relating  the  state  to  the  measurement  and  k  is  the  time  index. 

The  state  x{k)  is  available  only  as  an  estimate,  x(^|A:),  along  with  the  estimated  error 

covariance  matrix,  P(k\k).  The  estimate  A:)  of  the  state  at  time  k  is  based  on  all  the  past  data 
up  to  and  including  time  k.  The  estimate  is  available  from  a  suitable  estimator  like  an  extended 
Kalman  filter  or  maximum  likelihood  estimator.  The  residual  z  (k\k  - 1)  is  the  difference  between 
the  measurement  z(Jc)  and  the  estimate  of  the  measurement,  z  (A:|^  -  1),  that  is. 


z{k\k-  \)  =  z(k)-  z{k\k-  1), 


(B-2a) 


where 


zik\k-  l)  =  //[f(ifc|it- 1)], 


{B-2b) 


x{k\k-  l)  =  (Dx(yt-  1|^-  l)  +  5M(k- 1),  (B-2c) 

and  where  x(k\k  - 1)  is  the  state  estimate  at  time  k  based  on  the  measurements  up  to  and 
including  time  k  -  1  but  not  time  k.  The  process  to  be  investigated  here  is  the  predicted  residual 
sequence, 


(B-3a) 


where 


z.-z{k  +  /|A:), 


(B-3b) 


z{k  +  =  z{k  + 1)  -  j, 


(B-3c) 


^ O  Bu{k  +j- 1). 


(B-3d) 


Examining  the  /th  element  of  the  predicted  residual  sequence 

z{k  +  i\k)  =  H[x{k  +  /)]  -  H[xik  +  i\k)]  +  n{k  +  i), 
and  expanding  the  measurement  function  in  a  Taylor  series  about  the  estimate,  with 

dx{k  +  t)  =x(k  +  i\l()  -x(k  +  /), 
yields  the  approximate  expression 

z(A:  +  /|A:)  =  -a^[ic(A:  +  i\k)]Sx(k  +  ;)  +  n(k  +  i) , 
where 


a[x(k  +  /|ifc)]  = 


cHx{k  +  /■) 

|x(/:  +  /)  =  f(A:+/) 


(B-4) 

(B-5) 

(B-6a) 

(B-6b) 


Noting  that  the  dynamic  equation  for  the  propagation  of  the  state  estimation  error  in  terms  of  the 
fixed  error  at  time  k  is 

dx(k  +  i)  =  <t>iSx(k)  -  /(k  +  i)  W,  (B-7a) 

where 

/(k  +  0  =  [0)^-1  r,(D^-2r,...,or,r,o,...,o],  (B-7b) 

and 

W^=  [w(0),w(1),...,m/(;-2),w(;  -  1 ^(A^-l)].  (B-7c) 

The  entire  predicted  residual  sequence  can  be  approximated  as 

Z  =  -/4[x(A:|A:)](Sc(A:)  -GW+N„,,  (B-8a) 

where 
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t 


^  [x(A:|Jt)]  = 


^[xjk+i)] 

.  acik)  ) 


== 

a^[x(k  +  i\k)]F’ 

x(yfc)  =  x(A:|)t) 


(B-8b) 


and 


Nn  = 


n{k  +/) 

,G  = 

(B-8c) 


The  predicted  residual  sequence  can  be  approximated  as  a  jointly  Gaussian  random  vector.  For 
unbiased  state  estimates,  Z  is  zero  mean  and  its  probability  density  function  is 


det 


ZV  'Z  . 


(B-9a) 


Here,  V  is  the  covariance  matrix  of  the  predicted  residual  vector  defined  as 


cov(  Z )  =  £[zz^]  -  £[z]£[z'’], 

(B-9b) 

cov(  Z  )  =  E[{Nn  -  A  5x){Nn  -  A5x)^ . 

(B-9c) 

Thus, 

V=  AP(k\k)A^  +  ^  +  GQG^ 

(B-9d) 
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where 


Further,  it  is  noted  that  the  probability  density  function  for  the  predicted  residuals  is  only 
approximately  Gaussian.  However,  the  approximation  remains  valid  whenever  the  second-order 
and  higher  terms  of  the  Taylor  series  expansion  of  the  measurement  nonlinearity  can  be  ignored. 
Thus,  two  assumptions  on  the  predicted  residual  sequence  are  invoked  to  assure  the 
reasonableness  of  the  Taylor  series  approximation.  First,  the  dynamic  model  of  equation  (1), 
which  is  used  to  obtain  the  estimate,  must  be  a  reasonably  accurate  description  of  the  real  process. 
Second,  the  state  error,  and  its  effect  over  the  time  interval  of  interest,  must  remain  sufficiently 
small  to  allow  the  validity  of  a  first-order  Taylor  series  approximation  of  the  predicted  residual 
sequence.  The  two  assumptions  stated  above  are  the  basis  of  the  null  hypothesis,  Hq.  No 
modeling  error  and  a  good  state  estimate. 

Under  the  Ho  hypothesis,  the  predicted  residual  process  is  a  zero-mean  jointly  Gaussian 
random  vector;  that  is,  the  probability  density  function  is 


(B-ll) 


where  V  is  the  covariance  matrix  given  by  equation  (B-9d).  Note  that  although  elements  of  the 

predicted  residual  vector  are  correlated,  there  always  exists  a  set  of  basis  vectors  in  which  Z  can 
be  represented  by  uncorrelated  (and,  therefore,  independent)  components.  For  many  applications 
considered  here,  both  process  noise  and  state  estimation  error  covariances  are  small  compared 

with  the  measurement  noise  covariance.  For  these  cases,  the  elements  of  Z  are  approximately 
independent 

Under  the  alternate,  or  other  hypothesis,  H\,  there  is  a  modeling  error.  The  original  state  and 
measurement  models  of  equations  (B-1)  do  not  adequately  describe  the  system  dynamics.  As 
mentioned  in  the  introduction,  modeling  errors  may  arise  from  unknown  system  inputs,  a  change 
in  the  measurement  model,  or  an  unobserved  change  in  the  measurement  noise.  Examples  of  such 
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modeling  anomalies  are,  respectively,  a  target  maneuver,  change  in  propagation  path,  and 
measurement  bias.  Regardless  of  the  cause  of  the  modeling  error,  its  observable  effect  is  often  a 
feature  in  the  predicted  residual  sequence.  Under  hypothesis  /fothe  predicted  residual  process  is 
approximately  zero-mean  jointly  Gaussian,  and  for  small  state  error  with  no  process  noise,  is  a 
white  process. 

Under  the  mismodeling  hypothesis  Hi,  the  predicted  residuals  exhibit  features  that  are 
distinguishable  from  those  of  the  hypothesis  Hq.  For  the  hypothesis  H\  an  (l-l)-th  order 
polynomial  is  used  to  model  the  anomalous  features  of  the  predicted  residuals,  that  is 

Hi.  Z(k  +  ;|A:)  +  /|/f)j<5x(/f)  +  A77(/c  +  /)  +  /7(/f +  /)-g^(/f +  /)W,  (B-12a) 

where 


m{k  +  ;■)  =  [flto  +  aiitk+i  -  k)  +  ^2(4+,  -  k)  +  ai-i  (4+/  -  k) '  ]M(4+i  -  k), 

fO  /  <  11 

u[t\  =  <  ^  ^  ^  ,  and  k  is  the  time  of  the  modeling  anomaly. 

1  t  >  1 


(B-12b) 


The  predicted  residual  sequence  under  Hi,  is 


m.Z  =  -yl[x(^|A:)](5c(A:)  -GW+Mi+N„ 
where 


(B-13a) 


M\  =  Bm 


(B-13b) 


1  0  0 
"I  (4  ”  ^0 )  (4  “  ^0 )  •  •  • 


(B-13d) 


and 
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The  additive  term,  due  to  mismodeling,  M\,  contributes  to  the  mean  value  of  Z,  hence 


(B-13C). 


£[Z]=M, 


(B-14a) 


cov  E[Z\  =  E[iZ  -  Mi)(Z  -  A/i)1  =  V. 

Thus,  the  covariance  under  both  assumptions  is  the  same  and  only  the  means  differ.  The 
probability  density  function  for  the  jointly  Gaussian  random  vector  under  Hi  is 


(B-14b) 


p(z|//,)=  detV exp -j^{Z-M^)  V-'{Z-M,) 


(B-15) 


Under  hypothesis  Hi  (H\  :  A  modeling  anomaly  exists)  the  probability  density  function  of  the 
predicted  residuals  is  (possibly)  non-zero  mean  jointly  Gaussian, 


piZ\Hi)PN(Mi,V) 


(B-16) 


where  A/i  is  the  mean  vector  defined  in  equation  (B-13)  and  Fis  the  covariance  matrix  given  by 
equation  (B-9d),  and  N(M\,V)  is  the  Gaussian  probability  density  function  with  Mean  Mi,  and 
covariance  V. 

Similar  results  can  be  developed  for  both  the  smoothed  residuals  and  the  filtered  innovations. 
For  the  smoothed  residual  process  the  state  estimation  error  is  correlated  to  the  past  measurement 
noise.  Noting  that  the  conditional  probability  density  function  or  likelihood  function  for  the 
observations  (no  process  noise) 


Z  [Zi,...,Zn]  , 


(B-17) 


given  the  state  x  is. 


P{Z\x)^N{Z{x),S^) , 


(B-18a) 
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or 


P(Z\x)  =  det\S^\‘''^  \xp^-y,^{Z-Z{x)Y S-^{Z-Z{x^  ,  (B-18b) 

where 

Z(x)  =  [^x(l)],...,//[x(A0]].  (B-lSc) 

The  maximum  likelihood  estimate  is  the  value  of  the  state,  x,  that  maximizes  the  likelihood 
function  of  equation  (B-18).  Taking  the  logarithm  of  the  likelihood  function  yields  the  log- 
likelihood  equation. 

\n\p{Z\x)]  =  [constant]  +  -  Z(x))  ^ (Z-Z(x)).  (B- 1 9) 

Differentiating  the  log-likelihood  equation  with  respect  to  the  state  at  time  k  and  setting  the  result 
to  zero,  the  maximum  likelihood  estimate  can  be  obtained  by  solving  the  equation 

A\x{k)-\S-\Z  -  Z{x{k))\  =  0  (B-20) 

x(k)=x(/() 

For  the  linearized  smoothed  residuals 

Z,  =  -A[x{k\k))fixik)  +  N„  (B-21) 

wherei4[x(A|A:)]  is  defined  in  equation  (B-8).  The  error  in  the  estimate,  dx(k)  is  approximately 
dx{k)  »  [A^S-^A]  (B-22a) 

and  the  state  error  covariance  is 
P{k\k)=E{{dx{k)dx\k))}, 

(B-22b) 

which  can  be  approximated  as 

Pik\k)y>[A^S-^Ay\  (B-22c) 

Consequently,  from  equation  (B-9),  the  probability  density  function  for  the  smoothed  residuals 
under  Hq  is 

/>(Z|H.)W((r,),  (B-23a) 
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where 


K=cov(Z) 

or 

Vs  =  E[{N„-Adx)(N„-AdxY]- 
and,  substituting  from  equation  (B-22)  for  cbc(k)  yields 

Vs  =  ^+AP{k\k)A\  (B-23b) 

Similarly,  under  the  hypothesis  H\ 

(B-24) 

When  the  filtered  residual  sequence  is  obtained  from  the  extended  Kalman  filter,  the  estimate 
x{k/k)  is  the  conditional  mean,  and  the  filtered  residuals  or  innovations  process  is  white.^*  For  this 
process,  the  probability  density  function  is 

Ho:  P(  Zf\H,  )PN(0,  Vf)  (B-25a) 

Hr  P(Z^\H,)z^N(MuVf)  (B-25b) 

where 

Vf=  diag[a^P(i\j)a  +  s„^].  (B-25c) 

The  above  development  characterizes  three  processes.  They  are,  predicted  residuals, 
smoothed  residuals,  and  filtered  (conditional  mean)  residuals.  The  predicted  residuals  are 
obtained  from  future  measurements  and  the  state  prediction,  while  the  smoothed  residuals  are 
obtained  by  fitting  the  past  data  with  the  best  state  estimate.  The  filtered  residuals,  usually 
obtained  from  a  Kalman  filter,  involve  the  current  state  estimate  and  the  next  measurement. 

While  the  filtered  innovation  or  residuals  obtained  from  a  Kalman  filter  are  a  white  process,  both 
the  smoothed  and  predicted  residuals  are  correlated  to  some  degree;  however,  for  small  state 
estimation  error  they  can  often  be  approximated  as  a  white  process.  In  this  investigation, 
attention  is  focused  on  the  predicted  residuals,  and  for  some  results  is  limited  to  the  case  of 
negligible  state  estimation  error. 
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APPENDIX  C 

MULTIPLE  HYPOTHESIS  TEST 


In  the  previous  section,  the  problem  of  detecting  a  signal  with  known  structure  in  the 
presence  of  noise  was  considered.  Since  that  test  considers  one  of  two  possible  outcomes,  either 
the  signal  is  present  or  it  is  not,  it  is  often  called  a  binary  hypothesis  test.  This  section  discusses 
the  multiple  hypothesis  tests. 

The  anomalous  feature  model  was  defined  by  equation  (B-12b)  as  an  /th  order  polynomial. 

In  the  binary  hypothesis  test  of  the  previous  section  the  signal  detection  problem  was  considered. 
In  that  case  the  signal  structure  was  known,  that  is,  polynomial  order  was  known  as  well  as  which 
coefficients  are  non-zero.  Here,  only  the  maximum  model  order  is  known  and  a  test  is  devised  to 
determine  which  polynomial  coefficients  are  non-zero.  This  is  a  standard  problem  in  regression 
analysis  and  the  development  presented  here  is  adapted  from  Stuart  et  al.  and  Hoel  et  al.^^^^ 

The  case  considered  here  is  a  maximum  polynomial  order  of  one  for  the  anomalous  feature; 
that  is 

m  =  {ao,axf.  (C-1) 

The  hypotheses  are: 

Ho\  [ao  =  0,  ai  =.0]  no  anomaly  present  -  noise  only, 

H}  .  [ao  ^0,ai  =  0]  offset  or  jump  anomaly  only, 

H2'.  [do  =  0,  ai  0]  drift  -  only  anomaly,  and 
H3:  [ao  ^0,ai^  0]  jump  and  drift  anomaly 

The  development  is  easily  extended  to  arbitrary  model  order  by  noting  that  the  set  of  i 
polynomial  coefficients  results  in  2  ^  hypotheses. 

Consider  the  special  case  of  uncorrelated  error  in  the  predicted  residual  sequence.  This 
corresponds  to  the  case  of  small  or  no  state  estimation  error.  In  this  case  the  model  for  the 
predicted  residual  of  equation  (18-a)  of  the  text  is 

Z  =  N„+M.  (C-2) 

Since  the  measurement  noise  vector  N„  is  independent,  zero  mean,  Gaussian  random  vector,  it 
follows  that 

p(z)^n[m,sIi]. 

The  joint  density  function  of  the  predicted  residuals,  conditioned  on  the  hypothesis  Hi  that  the 
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/th  anomalous  feature  is  present  in  the  sequence  (i.e.,  jump-only,  drift-only,  jump-drift,  etc.)  is 
P(Z\H)PN[Musl  I, 

where  M  is  the  corresponding  mean  vector  induced  by  the  anomaly  of  the  ;th  hypothesis.  The 
likelihood  ratio  test  between  any  two  hypothesis,  //,  and  Hj,  is 


Hj 

p{z\h) 

1  > 

p(z\h] 

Hi 

(C-3) 


Here,  if  the  LRT  exceeds  the  threshold  /,  then  the  hypothesis,  Hj,  corresponding  to  the  presence  of 

the yth  feature  in  Z,  is  rejected.  Likewise,  if  the  LRT  'xs  below  threshold,  then  hypothesis  Hi  is 
rejected. 

The  case  where  the  measurement  noise  standard  deviation,  s„,  is  known,  and  the  anomaly, 

M=[m\  .  ntfif,  is  unknown  is  considered  first.  Substituting  for p{  Z\H)  for  the  general  case  results  in 


(C-4) 


{inY'^det  F  ^  exp-  )/(z  -  m)  F-'(z  -  m)  ^ ' 

LRT{Z)=^ - 4^ - ^ - -  1. 

{iTrf^det  F  ^  exp  -  -  M^.)1 


Taking  the  natural  logarithm  yields 


ijv[m7i(z)]=>/||z-a/,||;,+)/1z-mJ|;,  m 


(C-5) 


Since  Mis  unknown  its  maximum  likelihood  estimate  is  used  to  obtain  the  generalized 
likelihood  ratio  test  (GLRT).  The  maximum  likelihood  estimate  ofm  is 

m  =  (C-6a) 

and  the  estimate  of  predicted  residual  mean  M,  is 

M  =  Bm.  (C-6b) 
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2  .  A 

For  the  case  of  interest,  V  =  s^I  and  since  M  is  unknown,  the  estimateM  of  equation  (C-6b)  and 
(C-6a)  are  used  in  (C-5)  yielding  the  generalized  likelihood  ratio  test  (GLRT) 


Hj 

z-mP  z-mP  > 

\n[GLRT{Z)]  = - ^ - ^  ^  INA^ 

_< 

Hi 


(C-7a) 


or 


a 

k=\  cr„  k=\  < 

Hi 


\n^ , 


(C-7b) 


A  ^  A 

vihttemjk  is  the  kth  component  of  the  vectorA^’.  Define  the  test  statistic  for  the  multiple 
hypothesis  GLRT  as 

•.(z)=l«[GZ,7i7-(z)]  (C-7c) 

To  obtain  quantifiable  results  for  the  GLRT,  it  is  necessary  to  know  the  probability  density 
function  of  the  test  statistic  1„.  The  remainder  of  this  section  is  devoted  to  the  development  of  the 
test  statistic's  probability  density  function. 

From  equation  (C-2),  the  mean  vector  Mean  be  decomposed  into  a  linear  combination  of 


columns  of  the  matrix  B,  that  is 

M=ao,  B\  +aiB2  +  a ^.\Bi 

(C-8a) 

where  Bi  are  the  columns  of  B 

B=[B,\...\B{\ 

(C-8b) 

and 

w  =  [ao,...,a;.if. 

(C-8c) 

The  set  of  vectors  {Bj}  .  f  ^  are  independent  and  form  a  basis  of  an  i  -dimensional  subspace  of 

the  TV-dimensional  space  of  the  predicted  residual  vector.  Let  the  yth  hypothesis  be  the  anomaly 

modeled  by  the  6 order  polynomial  with  all  coefficients  present  (H2  for  the  case  of  interest). 
The  other  hypotheses  are  developed  as  constraints  on  the  coefficients,  that  is 
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C-o 


(C-9) 


For  the  case  of  interest  here,  the  hypothesis  //,  result  in  the  following  constraint  matrices,  C, 


^o:Co  =  /2x2;  //,;C,  =  [0,1];  //2:C2=[1,0] 

where  12x2  is  the  2x2  identity  matrix. 

When  hypothesis  Hi{\^j)  is  true,  /^-constraints  are  imposed  on  the  coefficients,  a,,  that  can  be 
used  to  reduce  the  number  of  independent  terms  in  the  decomposition  of  M  in  equation  (C-9). 
Consequently,  if Hi=  H2  then  ai  =  0, 


M^=aoBu  (C-10) 

where  the  rank  of  C  equals  p,  and  the  space  spanned  by  the  remaining  columns  of  B  has  been 
reduced  by  one.  In  general,  if  /^-constraints  are  imposed  by  the  /th  hypothesis,  then 

M,=aon,+,...,+af_p_,nf_p  ,  (C-11) 

where  a ,  are  the  surviving  coefficients  and  n,  are  the  basis  vectors  of  ( ^  -p)-dimensional  subspace 
spanned  by  M  under  //,. 

Let  the  set  of  m\  vectors  {r,}  be  an  orthonormal  basis  for  the  M-dimensional  space  of  the 
predicted  residuals.  Specifically,  let  the  first  (f  -p)  vectors,  {r,}  be  the  basis  for  the  space 
spanned  by  the  constrained  columns  of  B,  and  further,  let  the  set  of  vectors  {r;}  span  the 

column  space  of  B,  then 


Z  =  Ra,  (C-12a) 

and 

M=^Rg,  (C-12b) 

where, 

7?=[/-,|...|rAr]  (C-12c) 

and  q  and  g  are  (A6cl)-vectors  of  the  coefficients  of  Z  and  M,  respectively,  with  respect  to  the 
orthonormal  basis  vectors  {/•,}  and 

q=[au...,aN]^  (C-12d) 


and 
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Z=  [Y1.  •,Y/-p,  0,  ...of. 


(C-12e) 


Substituting  equation  (C-12)  into  the  two  forms  of  equation  (C-5),  under  the  condition  that  the 
test  hypothesis  //,  is  true,  results  in 


(C-13a) 


(C-13b) 


For  the  GLRT,  M  is  not  available  and  must  be  estimated.  The  estimate  is  given  by  equation 
(C-7),  which  for  the  case  of  interest  here  (V=  c/l)  is  equivalently  the  least-squares  estimate  or  the 
maximum  likelihood  estimate.  Since  the  estimate  of  M  must  minimized  the  sum  of  the  squared 

error  in  the  fit  of  the  mean,  M,  to  be  predicted  residuals,  Z,  it  follows  that  M  must  minimize 
equation  (C-13).  From  the  right-hand-side  of  equation  (C-13)  the  least  squares  estimate  of 
M  =  Ry\^ 


Hi:y,=a,;k^\ . t-p. 


(C-14a) 


Hj:y,=a^:k=^\,...J. 


(C-14b) 


Consequently,  equation  (C-13)  reduces  to 


iz-M,ir= 


(C-15a) 


t=/-p+l 


Substituting  equation  (C-15)  into  the  GLRT  of  equation  (C-7)  and  reducing,  yields 


(C-15b) 


L{2)  = 


^  a^k]  >  2 

k=i-p+\  ^  y  \ 


(C-16) 


The  predicted  residual  Z  is  a  Gaussian  random  vector  with  mean  Mi  and  covariance  rf /,  and 
probability  density  Sanction  (PDF)  given  by 
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p(Z)=>N{Mi,a‘I)^ 


(C-17) 


Using  equation  (C-13)  the  PDF  of  the  random  vector  a  is 

p{a)  =  [det\R-\]-^p{R-^^.  (C18) 

Since  ^  is  a  linear  orthogonal  transformation  (/?"'  =  and  det\R\  =  1),  it  follows  that  the  random 
vector  a  is  also  Gaussian  with  PDF, 

p(a)=>N(x,o^I).  (C19) 

Consequently,  the  GLRT  test  statistic  f  m  is  the  sum  of p  squared  zero  mean,  unit  variance, 
independent  Gaussian  random  variables  and  is,  therefore,  a  chi-squared  random  variable  with  p- 
degrees  (rank  of  the  constraint  matrix)  of  freedom.  Recall  that  this  result  was  obtained  under  the 
condition  that  /f,  is  true.  Therefore,  the  test  statistics,  £  «,  will  exceed  the  threshold  and  reject  Hi 
when  Hi  is  true  with  a  probability  that  is  given  by  the  chi-squared  distribution  with  p-degrees  of 
freedom.  Similarly,  if  Hj  is  true,  then  at  least  some  of  the  random  variables  {ak.  k  =  £-p,...  £} 
are  not  zero  mean  and  the  test  statistic  becomes  a  non-central  chi-squared  random  variable  with 
/7-degrees  of  freedom.  Under  the  conditions  of  Hj,  the  test  statistic  increases  and,  thereby, 
increases  the  probability  that  the  threshold  is  exceeded  and  HM  rejected. 

The  GLRT  test  statistic  can  be  represented  in  a  form  that  is  more  convenient  to  use.  Noting 
from  equation  (C-12)  and  (C-13)  that 

(C-20a) 

and 

(C-20b) 

k=\ 

it  follows  that  difference  in  the  estimates  of  the  means  under  hypothesis  j  and  /  is 

(C-20C) 

k-t-p+J 

and  the  norm  of  the  squared  distance  of  the  mean  is 

(c-21) 

k=e-p+i 

Note,  however,  that  the  right-handed  side  of  equation  (C-21)  is  the  GLRT  test  statistic;  hence 
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(C-23b) 


“0 

> 

To:  — f-  "  ^  .  (C-23c) 

O'n  < 

The  tests  T,  evaluate  the  statistical  significance  of  the  presence  of  each  coefficient  or  feature  in  the 
pol)niomial  indicative  of  anomalies  in  the  predicted  residual  sequence.  For  72  the  test  statistic  is 
the  norm-squared  value  in  units  of  noise  standard  deviation  of  the  difference  between  the 
estimates  of  the  mean  obtained  under  the  hypothesis  that  jump  and  drift  features  are  present 
(M3=>ao  0,  ai  0)  and  the  hypothesis  that  only  drift  is  present  (M2=>ao  =  0,ai^  0).  Thus,  T2  is 

a  test  of  the  significance  of  the  non-zero  estimate  of  the  jump  coefficient  Oo.  If  the  test  statistic  is 
below  threshold,  the  hypothesis  that  jump  and  drift  are  present  is  rejected  in  favor  of  the 
hypothesis  that  only  a  drift  feature  is  present  and  no  significant  jump  feature  is  evident. 
Conversely,  if  the  test  statistic  exceeds  the  threshold,  the  hypothesis  that  only  a  drift  is  present 
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is  rejected  in  favor  of  the  hypothesis  that  both  jump  and  drift  features  are  evident  in  the 
sequence.  When  H2  is  true,  which  implies  Qq  is  zero,  then  the  test  statistic  is  (central)  chi-squared 
with  one  degree  of  freedom.  Consequently,  the  threshold,  €2,  can  be  established  to  give  a  desired 
probability  of  false  detection  of  the  jump  feature  (ao  0).  When  is  true,  a  jump  is  present,  the 
jump  coefficient  a©  is  non-zero,  and  the  test  statistic  is  non-central  chi-squared  with  one-degree  of 
freedom.  Although  the  value  of  the  jump  coefficient  ao  is  unknown,  the  parameter  of  non¬ 
centrality  could  be  computed  for  some  value  of  ao  and  the  threshold  G2  set  to  achieve  a  specified 
probability  of  jump  detection  at  a  given  amplitude.  However,  this  approach  was  not  used.  Notice 
that  rejection  of  the  jump  plus  drift  hypothesis  (//s)  does  not  necessarily  imply  that  the  drift 
hypothesis  is  accepted.  The  only  conclusion  that  can  be  drawn  from  test  T2  is  whether  or  not  a 
jump  is  present.  Similar  comments  apply  for  test  T\  which  determines  if  a  drift  is  present.  The 
last  test,  T2,  determines  if  the  combined  hypothesis  of  jump  plus  drift  is  to  be  accepted/rejected 
over  the  null  hypothesis  of  no  anomaly,  Hq.  This  test  statistic  is  chi-squared  with  two  degrees  of 

freedom  when  Hq  is  true.  Test  To  is  redundant  with  tests  T\  and  T2  and  is  not  used. 

> 

To  complete  the  hypothesis  test,  a  logic  tree  is  constructed  from  the  following  table  of 
possible  outcomes. 


Table  C-1.  Multiple  Hypotheses  Test 


T2 

Ti 

(Oq  ^  0) 

H, 

(ao=  0) 

(a,  7^0) 

H, 

(ai=  0) 

If  the  result  of  T\  indicates  the  presence  of  a  drift  (reject  H\=>ai  0)  and  that  of  Tj  indicates  the 
absence  of  a  jump  (reject  H-i=>ao),  the  logical  conclusion  is  that  the  predicted  residuals  exhibit  a 
drift-only  feature. 
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APPENDIX  D 

FEATURE  AMPLITUDE  EVIDENCE  REPRESENTATIONS 


Evidence  pertaining  to  the  amplitude  of  a  feature  is  embedded  in  the  probability  density 
function  of  the  feature  estimates.  The  feature  estimates  are  obtained  from  equation  (22)  of  the 
text  and  the  density  function  approximated  by  a  Gaussian  density;  the  mean  value  of  the  density  is 
taken  a  the  estimated  value  and  the  variance  obtained  from  equation  (22).  Basic  probability 
assignments  for  the  feature  amplitude  frame  of  discernment  are  obtained  from  operations  on  the 
expected  value  of  the  feature  amplitudes  as  summarized  below.  Feature  amplitude  evidence 
representations  are  discussed  in  greater  detail  by  Ferkinhoff,  et  al.*'*  cited  in  the  text. 

Figure  D-1  illustrates  the  feature  density  function  (lower  portion)  and  the  feature  amplitude 
membership  functions.  The  first  step  is  to  compute  the  expected  for  each  of  the  feature 
amplitudes:  strong  negative  (s~),  moderate  negative  (w),  weak  negative  or  positive  (w*,  moderate 
positive  (ffJ*),  and  strong  positive  (s*). 


MfaJ 


Figure  D-1.  Feature  Amplitude  Regions  With  Polarity  Discrimination 
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The  expected  values  for  the  amplitudes  with  polarity  are  given  equations  D-1  and  D-2. 


-»-00 

E{w±  (a,)}  =  J  MJa,  )P(a,  )da, , 

-00 

(D-la) 

■f  00 

E[m„+  (ay)}  =  1  M„JaJP(aJda, , 

-00 

(D-lb) 

-foo 

£{/w„-(ay)}=  1  MJa,)P(aJda,, 

-00 

(D-lc) 

+  00 

£{/M,  +  (ay)}=  j  MJaJP(a,)da., 

-00 

(D-ld) 

+  00 

E{ms-  (a,) }  =  J  (a,  )P(a,  )da.  . 

(D-le) 

-00 


where  the  Mj  are  the  individual  membership  functions  for  weak,  moderate  positive,  moderate 
negative,  strong  positive,  and  strong  negative,  respectively. 

For  unimodal  density  functions  some  elements  of  the  power  set  for  the  feature  amplitude 
frame  of  discernment  with  polarity  result  in  illogical  conclusions.  For  example,  the  set  (strong 
negative,  strong  positive}  is  not  a  logical  possibility.  A  feature  has  either  a  strong  positive 
amplitude  or  a  strong  negative  amplitude,  but  not  both  simultaneously.  Similarly,  the  conditions 
of  (weak,  strong  positive}  cannot  exist.  Consequently,  a  reduced  frame  of  discernment  is 
considered  for  implementation  as  show  in  figure  D-2. 


{str + ,str-,mod + ,mod-,weak} 


{str + ,inod + ,mod-,weak}  {str-, mod + ,mod-,weak} 


Figure  D-2.  Revised  Power  Set  for  Discriminating  Polarity 
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The  basic  probability  assignments  for  the  elements  of  the  power  set  of  the  feature  amplitude  frame 
of  discernment  with  polarity  is  illustrated  in  figure  D-3. 


EXPECTED  BELIEF 


Figure  D-3.  Belief  Distribution  When  Discriminating  Polarity 


Here,  figure  D-3  illustrates  the  expected  values  of  the  feature  amplitudes  as  the  vertical  bars 
of  a  histogram.  The  BPA  is  determined  from  a  form  of  signal  excess  similar  to  that  for  feature 
existence  as  explained  in  the  text.  The  histogram  is  cut  at  the  various  values  for  the  expected 
values.  The  BPA  is  the  difference  in  levels  times  the  number  of  bars  in  the  histogram  that  are 
intersected.  The  resulting  value  of  BPA  is  assigned  to  the  union  of  the  sets  represented  in  the  cut. 
Thus,  for  the  example  illustrated  in  figure  D-3,  the  twice  the  difference  between  the  expected 
values  for  moderate  positive  feature  and  a  weak  feature  is  the  BPA  given  to  set  {weak,  moderate 
positive}.  The  equations  for  the  complete  assignment  of  BPA  to  the  reduced  frame  are  given 


below  in  equations  (D-2) 

7m/  =MAX[es*{a^  -  MA){[e„*{a),e„\a),es'{a^,ef(ai)] ,  0.0] ,  (D-2a) 

/nr  =  MAX[e;{a^  -  MAX[erf{a),e„\a^),es\a^),ef'{a^)] ,  0.0] ,  (D-2b) 

/n„*  =  MAX[e,f{ai)  -  MAX{e^-{a^,es*ia),e;{a^,ef'{a^, ,  0.0] ,  (D-2c) 

m„'  =  MAX[e„'{a^  -  ,  0.0] ,  (D-2d) 

mj  =  MAX[ef'(ai)  -  MAX{e„^(a),e^{a^,es\a),es'{a)'\ ,  0.0] ,  (D-2e) 

},0.0],  (D-2f) 

=MAY[2.0{M/AT;em^(a/),e/(a/)]-MlY[e/(a,),e„‘(a,),e/(a,)]},0.0],  (D-2g) 

m,;f=MAX[2D{MIN[e„\a),ef{a)]-MAX[es\a;),e„\^^^  (D-2h) 

ms~m~  =^MAX[2D{MIl^es\a),e„\a)yMAX[ef  (a),e„'^(a),es\a)]},0.0],  (D-2i) 
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rrisW^  =  MAY[3.0{M/ATe/(a),e„^(a),e„*(a)]-M^J^er(a),e„-(a)]},0.0],  (D-2j) 

m„\\-=MAX[3.Q{MIN[e„\a;),e„\a;),eJ^{a)]-MAX[eria^^^^  (D-2k) 

/nrmV=A/^^3.0{M/AIe/(a,),e«-(aO,e.*(a,)]-A^4^<(a^^^  (D-21) 

msW\-  =MAX[A.O{Mme:{a),e,:{a),e„\a^,e„\a)^^^^^  0.0] ,  (D-2m) 

ms-,:.\-=MAX[A.O{MIN[es\a;),e„\a),eJ^ia^,e^\^^^  0.0] ,  (D-2n) 

ms*m\'m's'  =  S.QMIN[es*{a),e„\a),eJ'{a^,e„-{a),es'ia^)] .  (D-2o) 

Since  the  amplitude  frame  does  not  distinguish  between  positive  and  negative  features  the 
final  BPA  assignments  are  without  regard  to  polarity  and  are  given  by  the  compatibility  relation 
shown  in  table  D-1. 
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Table  D-1.  Compatibility  Relation  Between  Feature  Amplitudes 
With  and  Without  Polarity 


Feature  Amplitude  Frame 
With  Polarity 

Feature  Amplitude  Frame 
Without  Polarity 

{/w,-jm,+jm,-js,+Js} 

{j^JrnJs) 

{jw-jm,  +jm,-is) 

{j^Jrnjs} 

{jw,-im,+jm„+js) 

{jwjmjs} 

{jw,-jmrjs] 

{jwjmjs) 

{jw,+jm,+js) 

{jwjmjs) 

{jw,-im,+im,) 

iM/w} 

{jw.-jm} 

{jw,jm} 

[jw,+jm} 

{jwjm) 

{-jm,-js} 

.  {jfnjs} 

{+jm,+js) 

imis) 

(/•s) 

{-» 

{/^} 

{.jm) 

m 

.  {+jm) 

m 

_ Itl _ 

_ itl _ 

Note  that  for  single  mode  probability  density  functions,  like  the  Gaussian,  logically 
inconsistent  assignments  of  BPA  to  sets  like  or  {Jw,-js}  cannot  occur  with  this 

technique.  Consequently,  non-zero  BPA  for  both  sets  {-js}  and  {+js}  also  cannot  occur 
simultaneously.  However,  a  non-zero  BPA  assignment  for  the  sets  {-js}  and  {-Js,-Jo}jw+Jtn,+Js} 
is  possible  and  does  not  create  a  logically  inconsistent  result. 

A  total  BPA  of  unity  in  the  frame  of  discernment  is  obtained  only  if  special  symmetry 
relations  are  maintained  with  the  functions  that  define  the  regions  of  feature  amplitudes  in  figure 
7b.  The  boundaries  for  the  feature  amplitude  regions  were  obtained  from  a  perturbation  analysis. 
The  perturbation  analysis  was  performed  numerically  with  a  monte  carlo  simulation  and  the 
results  presented  in  the  form  of  scatter  plots  of  event  occurrence  in  a  jump-drift  feature  plane. 

The  scatter  plots  are  discussed  in  appendix  E.  The  boundaries  were  established  by  visual 
inspection;  no  attempt  was  made  to  optimize  the  boundary  locations. 
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APPENDIX  E 

FEATURE  SCATTER  PLOTS 


Scatter  plots  were  used  to  determine  where  the  thresholds  for  the  features  of  strong, 
moderate,  and  weak  should  be  set  to  discriminate  among  the  models  for  target  maneuver,  change 
in  propagation  path,  change  in  base  frequency,  and  unknown  model  changes.  Additionally,  the 
scatter  plots  were  also  used  to  determine  the  compatibility  relationships.  The  scatter  plots  are 
shown  in  figures  E-1  through  E-4  for  the  conical  angle,  sphere  bearing  angle,  sphere 
depression/elevation  angle,  and  frequency,  respectively.  Only  one  frequency  plot  for  both  sensors 
is  presented  because  the  results  are  practically  identical. 

In  all  the  plots,  the  jump  amplitude  is  plotted  along  the  horizontal  axis;  the  drift  amplitude  is 
plotted  along  the  vertical  axis;  and  the  location  of  the  thresholds  have  been  indicated  with  solid 
lines.  The  individual  values  of  the  jump  and  drift  coefficients  are  indicated  on  the  plot  using  three 
different  symbols,  a  solid  square  for  contact  maneuver  (CM),  a  diamond  for  change  in 
propagation  path  (PP),  and  a  circle  for  a  change  in  the  base  frequency  (BF).  The  plots  were 
obtained  from  a  monte  carlo  simulation  involving  42,000  trials.  The  measurement  data  for  each 
trail  was  generated  from  a  randomized  single  leg  target-observer  geometry  with  23  measurement 
sets  at  20-second  intervals.  The  randomized  geometries  were  selected  using  a  uniform 
distribution  of  relative  contact  course  values  of  five  samples  between  0°  and  350°;  relative 
contact  speed  of  five  uniformly  selected  samples  between  2  and  15  yd/sec;  frequency  five  samples 
uniformly  selected  between  200  Hz  and  2  kHz;  depth  between  100  yd  and  10  kyd;  and  a  nominal 
sound  speed  of  1400  yd/sec.  The  model  anomalies  were  target  course  changes  of  ±170°  in  six 
divisions;  speed  change  of  ±10  yd/sec  in  six  divisions;  depth  changes  of  ±10  kyd  in  15  divisions; 
and  frequency  changes  of  0.5  percent  to  1.1  percent  in  15  divisions. 

For  each  of  the  three  types  of  model  anomalies,  CM,  PP,  BF,  14,000  monte  carlo  trials  were 
obtained.  For  each  monte  carlo  trial,  a  random  geometry  was  generated  and  the  noise-free 
predicted  residuals  were  computed.  The  linear  regression  fit  to  the  predicted  residuals  was 
obtained  yielding  the  jump  and  drift  coefficients.  The  values  of  the  jump  and  drift  coefficients 
were  quantized  and  placed  in  bins  and  accumulated,  by  bin,  across  the  trials.  All  the  bins  with 
non-zero  entries  are  shown  in  the  plots  E-1  through  E-4.  Note  that  in  these  scatter  plots,  bins 
with  one  entry  are  represented  equally  with  bins  that  have  thousands  of  entries.  The  thresholds 
are  set  empirically  using  visual  inspection  to  obtain  a  meaningful  separation  among  the  classes  of 
models.  No  attempt  was  made  to  optimize  this  the  selection  of  the  boundaries.  The  thresholds 
obtained  by  this  procedure  are  given  in  tables  within  the  plots  E-1  through  E4  for  the  azimuth 
bearing,  DE  angle,  conical  angle,  and  frequency,  respectively. 
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Figure  E-1.  Scatter  Plot  for  Conical  Angle 
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Figure  E-2.  Scatter  Plot  for  Sphere  Bearing  Angle 
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Figure  E-3.  Scatter  Plot  for  Sphere  Depression/Elevation  Angle 
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